qi`?TddlZddlZddlmZddlmZmZddlZddlm cm Z ddl m Z ddl mZddlmZmZmZmZmZmZmZddlmZddlmZdd lmZej:eZd ej@jBd e"ed e#d efdZ$Gdde jJZ&GddeZ'y)N)Callable)AnyUnion)ir)KernelTemplate)Buffer FixedLayoutget_free_symbolsget_symbolic_inputsgm_original_output_stridesir_node_to_tensorLayout) benchmarker)do_bench_using_profilingVgminputsnamereturnc ddlm}|||S)a.Inline a subgraph by converting its FX operations to individual IR nodes. This converts a subgraph to multiple ComputedBuffer nodes (fusable), enabling epilogue fusion with subsequent operations. Returns: TensorBox containing the final operation result as individual IR nodes r)process_subgraph_nodes)torch._inductor.loweringr)rrrrs f/home/ubuntu/crypto_trading_bot/.venv/lib/python3.12/site-packages/torch/_inductor/codegen/subgraph.pyinline_subgraph_to_ir_nodesrs@ !"f --c FeZdZdZdedeedededede fdd f fd Z defd Z d ee de e ee ffd Z d ee dejdefdZd ee dejdd fdZdefdZdeej,ej.ffdZdeee ffdZdefdZxZS)SubgraphChoiceCallerz Represents a Subgraph Autotuning choice, and the subgraph can be any arbitrary GraphModule. Compiles the Subgraph down to a module for benchmarking. r input_nodeslayout description make_fx_graph.rNcht|||||g|_tj5|j D]}t t|jddk(sJt t|jddk(sJ|jj|jjt| ddd||j|_t|jt!|j |_d|_d|_y#1swY[xYw)NT) unbacked_onlyr)super__init__example_inputsr fake_moderlenr get_size get_stridedata freeze_layoutappendr rr r sym_inputs_compiled_module_compiled_sym_inputs)selfrrr r!r"inp __class__s rr&zSubgraphChoiceCaller.__init__1s  {FK@  [[ C'' C+CLLN$OPTUUUU+CNN,.>?&*6:!! C Cs B"D((D1c"d|jdS)NzSubgraphCaller()rr2s r__str__zSubgraphChoiceCaller.__str__Ns  1--rargsc ~ddlmcm}ddlm}|j j ddj dd}||j|jtjjtjjtjjtjjtjjtjj d| }|j"D]@}||j$|j <|j&j)|j B|j"Dcgc]>}t+tjj,j.j1|@}}t3|dk(rt5||jD]z\} } t7| t8j:s!t7| t8j:sJ| j<| j<k(sJ| j?| j?k(rzJtj@|5|jCd d d 5|jD|j|jG} dddddd |fScc}w#1swYxYw#1swY |fSxYw) z Compile the subgraph for benchmarking and return (module, sym_inputs). TODO: Add precompile() method to enable parallel compilation of all choices before benchmarking. rN) GraphLoweringz::_. benchmark_) rr' shape_env cpp_wrapperaot_modeextern_node_serializer is_inference is_backwardrFATEN) max_autotunemax_autotune_gemmmax_autotune_gemm_backends)$torch._inductor.config _inductorconfigtorch._inductor.graphr<rreplacerr'rgraph _shape_envrArBrCrDrEr/ graph_inputsgraph_input_namesr.intsizevarsr@ size_hintr)zip isinstancetorchTensorshapestrideset_graph_handlerpatchruncompile_to_module) r2r:inductor_configr< safe_namebm_graph_loweringsym_inpsym_varr/ar example_inpmods r_compile_for_benchmarkingz.SubgraphChoiceCaller._compile_for_benchmarkingQsI 987II%%dC088cB )ww..gg((++WW%%#$77#A#A--++i[)   EG;B  * *7<< 8  / / 6 6w|| D E ??    **44W= >  z?a $'tT-@-@#A ?Kb%,,/%k5<<@@@88{'8'888899;+*<*<*>>>>  ? !2 3 < &&""'+1' < &!%%t':':;'99;  < <J7 & < < <Js+AJ J0!*J$ J0$J- )J00J<outcT|j!|j\}||_|_n|j}|jJ|jtj rt fdStjfdtjgS)zT Regular benchmarking: compile and use benchmarker with warmup/rep. cgSNr:bm_funcr/srz0SubgraphChoiceCaller.benchmark..sG.sG0j0401r)device) r0rhr1callrL/profile_bandwidth_with_do_bench_using_profilingrr benchmark infer_device)r2rir:rgror/s ` @@rruzSubgraphChoiceCaller.benchmarks  (=R7r tCy u||  ,&$s)&%,,&4&$ #   U2<<1I1I#IJ   4S> '#'rrceZdZdZej Zdeffd Z ddede e de de de fded e d efd Z dded e e de fde e de eee fde de fd zd e ef dZde de fd eee fd efdZd eee fd d fdZded e e de fde e d d fdZ dde e de de fd eee fde de fd zd e f dZxZS)SubgraphTemplatea  A template for subgraph evaluation to be used in autotuning. This class allows creating customized subgraphs that can be appended as choices during the autotuning process, enabling the selection of optimal implementations for complex operations. rc&t||y)z Initialize a subgraph template. Args: name: The name of this template graph: The FX graph r7N)r%r&)r2rr4s rr&zSubgraphTemplate.__init__s d#rrr r".r!kwargsrc Zt|dttj||||S)a- Generate a SubgraphChoiceCaller instance for autotuning. Args: name: The name for this subgraph choice input_nodes: List of input nodes to the subgraph layout: Memory layout information for the output make_fx_graph: Callable that creates the FX graph for this subgraph description: Optional description of this choice **kwargs: Additional keyword arguments Returns: SubgraphChoiceCaller: A callable object that can be used for autotuning r=)rrr r!r")rnextr index_counter)r2rrr r"r!rs rgeneratezSubgraphTemplate.generates80$64 0 > >?@A##'   rNdecompositionsnon_tensor_args default_implc T|sgSt|t|k(s Jdt|dt|dt||Dcgc]\}}|j||||}}}|j||||d} g} t||D]\}} ddl|| ddt dt d t fd ttt fd t ffd } |j|| } |j|d | || | d|j}| j|| Scc}}w)a Generate multiple SubgraphChoiceCaller instances for custom op autotuning. This method extends SubgraphTemplate to support custom op decompositions, allowing multiple implementations to compete in autotuning. Args: name: Base name for the choices decompositions: List of decomposition functions to compete in autotuning input_nodes: List of tensor inputs. All tensor arguments must be passed here. non_tensor_args: List of non-tensor kwargs only, one dict per corresponding decomposition. default_impl: Default implementation for layout inference Returns: List of SubgraphChoiceCaller instances for autotuning z>decompositions and non_tensor_args must have same length, got z decompositions and z kwargsrN)decomp decomp_kwargsr:r.rrcfddlm}ddlm}|}|j|fi|||S)Nr)make_fx)select_decomp_table)decomposition_table)"torch.fx.experimental.proxy_tensorr decompositionrpartial)rrr:rrr functoolss rr"zBSubgraphTemplate.generate_custom_op_choices..make_fx_graph9sH G?&9&;#w%I%%f> >(;rr=z CustomOp )rrr r"r!) r)rV_infer_custom_op_layout_validate_layout_equivalencerrrrr~_generate_variant_namerrr.)r2rrrrrrrlayoutsr choicesrr" variant_namechoicers @rgenerate_custom_op_choicesz+SubgraphTemplate.generate_custom_op_choices sx0I>"c/&:: ~&'';Cz:SubgraphTemplate._generate_variant_name.._sNtq!1#Qqc Ns)rr|sorteditems)r2rr base_name param_suffixs rrz'SubgraphTemplate._generate_variant_nameXsFOO  xxNvflln7MNN Al^,,rc|jD]<\}}t|tjtfs'Jd|dt |dy)z8Validate that kwargs contains only non-tensor arguments.zkwargs['z'] contains tensor zo. Tensor arguments should be in input_nodes, not kwargs. Only scalar/non-tensor parameters should be in kwargs.N)rrWrXrYrtype)r2rkeyvalues r_validate_non_tensor_kwargsz,SubgraphTemplate._validate_non_tensor_kwargsbsZ ,,. JC!%%,,)?@ 3%24;-@IJ @ rop_namerc,|sy|d}t|dddD]\}}|j|j|j|jf|j|j|j|jfk7sdt d|d||j d|jd|jd|jd|jd |dj d|jd|jd|jd|jd y) zXEnsure all layouts have consistent stride, device, dtype, and sizes for fair autotuning.Nrr{)startzLayout mismatch in custom op 'z': decomposition 'z ' produces (z, z) but 'r6) enumeraterrdtypesizer[AssertionErrorr)r2rrr referenceir s rrz-SubgraphTemplate._validate_layout_equivalenceks( AJ "712;a8 IAv v||V[[&--H    M %4WI>&&4Q&7&@&@%AB bb R W*1-6678!(()IOO+NbQZQaQaPbbc e rfunction_decompositioncddl}ddlm}|j||j5g}|D]}|j } |j jj| tj} tj| |j|j} |j| |j |fi|} | |} t#| tj$sJdt'| dt)| j*| j,| j.| j1cdddS#1swYyxYw) zInfer output layout for custom ops using the default implementation when available. Note that the Subgraph assumes custom ops return exactly one tensor output. TODO: Add support for multiple output custom ops. rNr)fallback)rrrz#Expected single tensor output, got z:. Multi-output custom ops not yet supported in autotuning.)rrrrr[)rtorch._inductor.virtualizedrrr(r*rOrT size_hintsrLunbacked_symint_fallbackrXempty get_dtype get_devicer.rrWrYrr rrrrZr[)r2rrrrrrr'r3 raw_shapeconcrete_shape fake_tensorfnoutputs rrz(SubgraphTemplate._infer_custom_op_layouts4 1 ((0 [[ N" 3LLN !"!1!1!37 ,&\,!)c 2,S#X , sCx(4/ ,  ,rr)(rloggingcollections.abcrtypingrrrXrJrKrLtorch._inductorrtorch._inductor.codegen.commonrtorch._inductor.irrr r r r r r$torch._inductor.runtime.benchmarkingrtorch._inductor.utilsrrr getLoggerrlogfx GraphModulerr~r ChoiceCallerrrrmrrrs$ ''9=:)g! .  .&*3i .7: . . d'2??d'N_~_r