{"id":111606,"date":"2026-08-15T21:49:56","date_gmt":"2026-08-15T21:49:56","guid":{"rendered":"https:\/\/youzum.net\/fine-tuning-tool-calling-llms-a-complete-guide-using-xyz-aquila-sft-and-qwen3\/"},"modified":"2026-08-15T21:49:56","modified_gmt":"2026-08-15T21:49:56","slug":"fine-tuning-tool-calling-llms-a-complete-guide-using-xyz-aquila-sft-and-qwen3","status":"publish","type":"post","link":"https:\/\/youzum.net\/es\/fine-tuning-tool-calling-llms-a-complete-guide-using-xyz-aquila-sft-and-qwen3\/","title":{"rendered":"Fine-Tuning Tool-Calling LLMs: A Complete Guide Using XYZ-Aquila-SFT and Qwen3"},"content":{"rendered":"

In this tutorial, we implement an end-to-end supervised fine-tuning pipeline for the XYZ-Aquila-SFT<\/strong><\/a> dataset, Hugging Face Transformers, PyTorch, and PEFT. We stream and inspect the dataset, parse multi-turn tool-use trajectories, extract structured tool calls, analyze corpus characteristics, and preserve embedded reasoning and observation patterns. We then convert tool schemas between message-embedded and structured formats, render Qwen-compatible ChatML with assistant-only loss masking, prepare a custom PyTorch dataset and collator, and fine-tune Qwen3-0.6B with LoRA. Finally, we evaluate tool-call prediction before and after training and export both the transformed dataset and corpus statistics for further experimentation.<\/p>\n

\n
\n
\n
\n
<\/div>\n
<\/div>\n
<\/div>\n<\/div>\n
Copy Code<\/span>Copied<\/span>Use a different Browser<\/span><\/a><\/div>\n<\/div>\n
import os, sys, subprocess\nCFG = dict(\n   REPO            = \"XYZAILab\/XYZ-Aquila-SFT\",\n   LANG            = \"en\",\n   N_STREAM        = 400,\n   N_EVAL          = 40,\n   MODEL_ID        = \"Qwen\/Qwen3-0.6B\",\n   MAX_SEQ_LEN     = 2048,\n   LENGTH_POLICY   = \"truncate\",\n   RUN_TRAINING    = True,\n   MAX_STEPS       = 30,\n   GRAD_ACCUM      = 8,\n   LR              = 1e-4,\n   LORA_R          = 16,\n   RUN_EVAL        = True,\n   N_EVAL_PROBES   = 24,\n   OUT_DIR         = \"\/content\/aquila_out\",\n   SEED            = 0,\n)\nos.makedirs(CFG[\"OUT_DIR\"], exist_ok=True)\ndef pip(*pkgs):\n   subprocess.run([sys.executable, \"-m\", \"pip\", \"install\", \"-q\", \"-U\", *pkgs], check=False)\npip(\"datasets>=3.0.0\", \"transformers>=4.51.0\", \"peft>=0.13.0\", \"accelerate>=1.0.0\")\nimport json, re, math, random, statistics as stats\nfrom collections import Counter, defaultdict\nfrom dataclasses import dataclass, field\nfrom typing import Any, Dict, List, Optional\nimport torch\nimport matplotlib.pyplot as plt\nfrom datasets import load_dataset\nfrom transformers import AutoTokenizer, AutoModelForCausalLM, get_cosine_schedule_with_warmup\nrandom.seed(CFG[\"SEED\"]); torch.manual_seed(CFG[\"SEED\"])\nDEV = \"cuda\" if torch.cuda.is_available() else \"cpu\"\nBF16 = DEV == \"cuda\" and torch.cuda.is_bf16_supported()\nprint(f\"device={DEV}  bf16={BF16}  torch={torch.__version__}\")\nprint(f\"n[1] streaming {CFG['REPO']}:{CFG['LANG']} ...\")\nstream = load_dataset(CFG[\"REPO\"], CFG[\"LANG\"], split=\"train\", streaming=True)\nRAW: List[Dict[str, Any]] = list(stream.take(CFG[\"N_STREAM\"]))\nprint(f\"    pulled {len(RAW)} rows; keys = {list(RAW[0].keys())}\")\n_r = RAW[0]\nprint(f\"    question[:110]        : {_r['question'][:110]}...\")\nprint(f\"    answer                : {_r['answer'][:80]}\")\nprint(f\"    number of tool calls  : {_r['number of tool calls']}\")\nprint(f\"    trajectory len        : {len(_r['trajectory'])} msgs\")\nprint(f\"    role sequence (first8): {[m['role'] for m in _r['trajectory'][:8]]}\")\n<\/code><\/pre>\n<\/div>\n<\/div>\n
We configure the dataset, model, training parameters, output directory, and reproducibility settings for the complete workflow. We install the required Hugging Face, PEFT, Accelerate, and PyTorch-related dependencies and detect whether a CUDA GPU and BF16 support are available. We then stream a limited number of XYZ-Aquila-SFT examples, inspect the dataset schema, and examine the structure of the first tool-use trajectory.<\/p>\n
\n
\n
\n
\n
<\/div>\n
<\/div>\n
<\/div>\n<\/div>\n
Copy Code<\/span>Copied<\/span>Use a different Browser<\/span><\/a><\/div>\n<\/div>\n
TOOLS_BLOCK_RE = re.compile(r\"<tools>s*(.*?)s*<\/tools>\", re.S)\nTHINK_RE       = re.compile(r\"<think>(.*?)<\/think>\", re.S)\nTOOL_RESP_RE   = re.compile(r\"<tool_response>s*(.*?)s*<\/tool_response>\", re.S)\nTOOLS_HDR_RE   = re.compile(r\"nn# Toolsnn\")\ndef iter_json_objects(text: str, limit: int = 1):\n   \"\"\"Nesting-safe JSON scanner. Regex like r'\\{.*?\\}' breaks on nested\n   `arguments` objects, which every real tool call has.\"\"\"\n   dec, i, n, out = json.JSONDecoder(), 0, len(text), []\n   while i < n and len(out) < limit:\n       while i < n and text[i] not in \"{[\":\n           i += 1\n       if i >= n:\n           break\n       try:\n           obj, end = dec.raw_decode(text, i)\n       except json.JSONDecodeError:\n           i += 1\n           continue\n       out.append(obj); i = end\n   return out\ndef parse_tool_calls(content: str) -> List[Dict[str, Any]]:\n   calls = []\n   for m in re.finditer(r\"<tool_call>\", content):\n       got = iter_json_objects(content[m.end():], limit=1)\n       if got:\n           calls.append(got[0])\n   return calls\n@dataclass\nclass Trajectory:\n   question: str\n   answer: str\n   declared_calls: int\n   messages: List[Dict[str, str]]\n   system_core: str = \"\"\n   tools: List[Dict[str, Any]] = field(default_factory=list)\n   tools_suffix: str = \"\"\n   calls: List[Dict[str, Any]] = field(default_factory=list)\n   n_observations: int = 0\n   n_think: int = 0\n   @property\n   def tool_names(self):  return [c.get(\"name\", \"?\") for c in self.calls]\n   @property\n   def depth(self):       return len(self.messages)\ndef parse_row(row: Dict[str, Any]) -> Trajectory:\n   msgs = [{\"role\": m[\"role\"], \"content\": m[\"content\"]} for m in row[\"trajectory\"]]\n   t = Trajectory(row[\"question\"], row[\"answer\"], row[\"number of tool calls\"], msgs)\n   if msgs and msgs[0][\"role\"] == \"system\":\n       sysmsg = msgs[0][\"content\"]\n       split = TOOLS_HDR_RE.search(sysmsg)\n       if split:\n           t.system_core   = sysmsg[:split.start()]\n           t.tools_suffix  = sysmsg[split.start():]\n       else:\n           t.system_core = sysmsg\n       blk = TOOLS_BLOCK_RE.search(sysmsg)\n       if blk:\n           t.tools = iter_json_objects(blk.group(1), limit=64)\n   for m in msgs:\n       if m[\"role\"] == \"assistant\":\n           t.calls += parse_tool_calls(m[\"content\"])\n           t.n_think += len(THINK_RE.findall(m[\"content\"]))\n       else:\n           t.n_observations += len(TOOL_RESP_RE.findall(m[\"content\"]))\n   return t\nTRAJ = [parse_row(r) for r in RAW]\nt0 = TRAJ[0]\nprint(f\"n[2] parsed {len(TRAJ)} trajectories\")\nprint(f\"    tool schemas found : {[fn.get('function', fn).get('name') for fn in t0.tools]}\")\nprint(f\"    parsed calls       : {len(t0.calls)}  (declared {t0.declared_calls})\")\nprint(f\"    observations       : {t0.n_observations}   think blocks: {t0.n_think}\")\nif t0.calls:\n   print(f\"    sample call        : {json.dumps(t0.calls[0], ensure_ascii=False)[:200]}\")\nagree = sum(len(t.calls) == t.declared_calls for t in TRAJ)\nprint(f\"    parser vs 'number of tool calls': {agree}\/{len(TRAJ)} exact match\")\ncalls_per   = [len(t.calls) for t in TRAJ]\ndepth_per   = [t.depth for t in TRAJ]\nchars_per   = [sum(len(m[\"content\"]) for m in t.messages) for t in TRAJ]\nname_freq   = Counter(n for t in TRAJ for n in t.tool_names)\nargkey_freq = defaultdict(Counter)\nfor t in TRAJ:\n   for c in t.calls:\n       args = c.get(\"arguments\", {})\n       if isinstance(args, dict):\n           for k in args: argkey_freq[c.get(\"name\", \"?\")][k] += 1\ndef q(xs, p):\n   xs = sorted(xs); return xs[min(len(xs) - 1, int(p * len(xs)))]\nprint(\"n[3] corpus statistics\")\nprint(f\"    tool calls \/ traj : mean {stats.mean(calls_per):.1f}  p50 {q(calls_per,.5)}  \"\n     f\"p90 {q(calls_per,.9)}  max {max(calls_per)}\")\nprint(f\"    messages \/ traj   : mean {stats.mean(depth_per):.1f}  p90 {q(depth_per,.9)}  max {max(depth_per)}\")\nprint(f\"    chars \/ traj      : mean {stats.mean(chars_per):,.0f}  p90 {q(chars_per,.9):,}\")\nprint(f\"    tool distribution : {dict(name_freq)}\")\nfor k, v in argkey_freq.items():\n   print(f\"      {k:<24} arg keys -> {dict(v.most_common(6))}\")\ntot = sum(chars_per); top = sum(sorted(chars_per)[-max(1, len(chars_per)\/\/10):])\nprint(f\"    top-10% longest trajectories hold {100*top\/tot:.1f}% of all characters\")\nfig, ax = plt.subplots(1, 3, figsize=(15, 3.6))\nax[0].hist(calls_per, bins=40); ax[0].set_yscale(\"log\"); ax[0].set_title(\"tool calls \/ trajectory\")\nax[1].hist(depth_per, bins=40); ax[1].set_yscale(\"log\"); ax[1].set_title(\"messages \/ trajectory\")\nax[2].bar(list(name_freq), list(name_freq.values())); ax[2].set_title(\"tool usage\"); ax[2].tick_params(axis=\"x\", rotation=20)\nplt.tight_layout(); plt.show()\n<\/code><\/pre>\n<\/div>\n<\/div>\n
We define nesting-safe utilities for extracting JSON tool calls, reasoning blocks, observations, and embedded tool schemas from each conversation. We convert every raw dataset row into a structured trajectory object and verify that the parsed tool-call counts match the values declared by the dataset. We then calculate corpus-level statistics and visualize the distributions of tool calls, message depth, trajectory size, and tool usage frequency.<\/p>\n
\n
\n
\n
\n
<\/div>\n
<\/div>\n
<\/div>\n<\/div>\n
Copy Code<\/span>Copied<\/span>Use a different Browser<\/span><\/a><\/div>\n<\/div>\n
QWEN3_TOOLS_TMPL = (\n   \"You are provided with function signatures within <tools><\/tools> XML tags:n<tools>n\"\n   \"{lines}n<\/tools>nnFor each function call, return a json object with function name \"\n   \"and arguments within <tool_call><\/tool_call> XML tags:n<tool_call>n\"\n   '{{\"name\": <function-name>, \"arguments\": <args-json-object>}}n<\/tool_call>'\n)\ndef extract_tools(t: Trajectory) -> Dict[str, Any]:\n   \"\"\"message-embedded schemas -> {'messages': [...], 'tools': [...]}\"\"\"\n   msgs = [dict(m) for m in t.messages]\n   if msgs and msgs[0][\"role\"] == \"system\":\n       msgs[0][\"content\"] = t.system_core\n   return {\"messages\": msgs, \"tools\": t.tools,\n           \"question\": t.question, \"answer\": t.answer}\ndef render_tools(rec: Dict[str, Any]) -> List[Dict[str, str]]:\n   \"\"\"inverse: structured tools -> schemas re-embedded in the system message\"\"\"\n   msgs = [dict(m) for m in rec[\"messages\"]]\n   if rec[\"tools\"] and msgs and msgs[0][\"role\"] == \"system\":\n       lines = \"n\".join(json.dumps(x, ensure_ascii=False) for x in rec[\"tools\"])\n       msgs[0][\"content\"] = msgs[0][\"content\"] + QWEN3_TOOLS_TMPL.format(lines=lines)\n   return msgs\n_rt = render_tools(extract_tools(t0))\nexact = _rt[0][\"content\"] == t0.messages[0][\"content\"]\nprint(f\"n[4] extract->render byte-exact: {exact}\")\nif not exact:\n   print(\"    template drift detected -> using verbatim tools_suffix for render()\")\n   a, b = t0.messages[0][\"content\"], _rt[0][\"content\"]\n   i = next((i for i in range(min(len(a), len(b))) if a[i] != b[i]), min(len(a), len(b)))\n   print(f\"    first divergence @{i}: {a[i:i+70]!r}  vs  {b[i:i+70]!r}\")\ntok = AutoTokenizer.from_pretrained(CFG[\"MODEL_ID\"])\nif tok.pad_token is None:\n   tok.pad_token = tok.eos_token\nIM_START, IM_END, NL = \"<|im_start|>\", \"<|im_end|>\", \"n\"\ndef render_and_mask(t: Trajectory, max_len: int, policy: str):\n   \"\"\"Manual ChatML so we control masking token-exactly.\n   WHY NOT apply_chat_template(): Qwen3's template deletes <think>...<\/think>\n   from every assistant turn except the last. On this dataset that silently\n   destroys most of the reasoning supervision you are paying to train on.\n   \"\"\"\n   ids, labels = [], []\n   for m in t.messages:\n       head = tok(f\"{IM_START}{m['role']}{NL}\", add_special_tokens=False).input_ids\n       body = tok(m[\"content\"], add_special_tokens=False).input_ids\n       tail = tok(f\"{IM_END}{NL}\", add_special_tokens=False).input_ids\n       seg  = head + body + tail\n       if m[\"role\"] == \"assistant\":\n           lab = [-100] * len(head) + body + tail\n       else:\n           lab = [-100] * len(seg)\n       ids += seg; labels += lab\n   if len(ids) > max_len:\n       if policy == \"drop\":\n           return None\n       ids, labels = ids[:max_len], labels[:max_len]\n   if all(l == -100 for l in labels):\n       return None\n   return {\"input_ids\": ids, \"labels\": labels}\n_probe = [{\"role\": \"system\", \"content\": \"S\"}, {\"role\": \"user\", \"content\": \"U\"},\n         {\"role\": \"assistant\", \"content\": \"A\"}]\n_mine = \"\".join(f\"{IM_START}{m['role']}{NL}{m['content']}{IM_END}{NL}\" for m in _probe)\n_theirs = tok.apply_chat_template(_probe, tokenize=False, add_generation_prompt=False)\nprint(f\"n[5] manual ChatML == chat_template on tool-free probe: {_mine == _theirs}\")\nif _mine != _theirs:\n   print(f\"    mine  : {_mine!r}n    theirs: {_theirs!r}  (informational only)\")\nENC = [e for e in (render_and_mask(t, CFG[\"MAX_SEQ_LEN\"], CFG[\"LENGTH_POLICY\"]) for t in TRAJ) if e]\nsup = [sum(1 for x in e[\"labels\"] if x != -100) \/ len(e[\"labels\"]) for e in ENC]\nprint(f\"    encoded {len(ENC)}\/{len(TRAJ)} examples\")\nprint(f\"    supervised-token ratio: mean {stats.mean(sup):.3f}  p10 {q(sup,.1):.3f}  p90 {q(sup,.9):.3f}\")\nover = sum(1 for t in TRAJ if sum(len(tok(m['content'], add_special_tokens=False).input_ids)\n                                 for m in t.messages[:3]) > CFG[\"MAX_SEQ_LEN\"])\nprint(f\"    trajectories whose first 3 msgs alone exceed MAX_SEQ_LEN: {over}\")\nSPLIT = len(ENC) - min(CFG[\"N_EVAL\"], len(ENC)\/\/5)\nTRAIN_ENC, EVAL_TRAJ = ENC[:SPLIT], TRAJ[SPLIT:]\nclass SFTSet(torch.utils.data.Dataset):\n   def __init__(self, rows): self.rows = rows\n   def __len__(self):        return len(self.rows)\n   def __getitem__(self, i): return self.rows[i]\ndef collate(batch):\n   L = max(len(b[\"input_ids\"]) for b in batch)\n   pad = tok.pad_token_id\n   return {\n       \"input_ids\":      torch.tensor([b[\"input_ids\"] + [pad]*(L-len(b[\"input_ids\"])) for b in batch]),\n       \"labels\":         torch.tensor([b[\"labels\"]    + [-100]*(L-len(b[\"labels\"]))   for b in batch]),\n       \"attention_mask\": torch.tensor([[1]*len(b[\"input_ids\"]) + [0]*(L-len(b[\"input_ids\"])) for b in batch]),\n   }\nloader = torch.utils.data.DataLoader(SFTSet(TRAIN_ENC), batch_size=1, shuffle=True, collate_fn=collate)\nprint(f\"n[6] train={len(TRAIN_ENC)}  eval_trajectories={len(EVAL_TRAJ)}\")\n<\/code><\/pre>\n<\/div>\n<\/div>\n
We extract embedded tool definitions into a structured format and reconstruct them to test whether the conversion preserves the original system message. We manually render each trajectory in ChatML format to retain all reasoning content and apply loss only to assistant-generated tokens. We also tokenize the examples, enforce the selected sequence-length policy, create the training and evaluation split, and prepare a padded PyTorch DataLoader.<\/p>\n
\n
\n
\n
\n
<\/div>\n
<\/div>\n
<\/div>\n<\/div>\n
Copy Code<\/span>Copied<\/span>Use a different Browser<\/span><\/a><\/div>\n<\/div>\n
def build_probes(trajs, n):\n   \"\"\"Teacher-forced probes: cut the trajectory right before an assistant turn\n   that issues a tool call; the gold label is that call.\"\"\"\n   probes = []\n   for t in trajs:\n       for i, m in enumerate(t.messages):\n           if m[\"role\"] != \"assistant\":\n               continue\n           gold = parse_tool_calls(m[\"content\"])\n           if not gold:\n               continue\n           prefix = \"\".join(f\"{IM_START}x['role']{NL}\" for x in [])\n           prefix = \"\".join(f\"{IM_START}{p['role']}{NL}{p['content']}{IM_END}{NL}\"\n                            for p in t.messages[:i]) + f\"{IM_START}assistant{NL}\"\n           if len(tok(prefix, add_special_tokens=False).input_ids) > CFG[\"MAX_SEQ_LEN\"] - 160:\n               continue\n           probes.append({\"prefix\": prefix, \"gold\": gold[0]})\n           break\n       if len(probes) >= n:\n           break\n   return probes\n@torch.no_grad()\ndef eval_tool_calls(model, probes, tag):\n   model.eval()\n   name_hit = arg_f1 = parsed = 0\n   for p in probes:\n       enc = tok(p[\"prefix\"], return_tensors=\"pt\", add_special_tokens=False).to(model.device)\n       out = model.generate(**enc, max_new_tokens=160, do_sample=False,\n                            pad_token_id=tok.pad_token_id)\n       gen = tok.decode(out[0][enc.input_ids.shape[1]:], skip_special_tokens=True)\n       pred = (parse_tool_calls(gen) or iter_json_objects(gen, limit=1) or [None])[0]\n       if not isinstance(pred, dict):\n           continue\n       parsed += 1\n       g = p[\"gold\"]\n       name_hit += int(pred.get(\"name\") == g.get(\"name\"))\n       pk = set((pred.get(\"arguments\") or {}).keys()) if isinstance(pred.get(\"arguments\"), dict) else set()\n       gk = set((g.get(\"arguments\") or {}).keys())    if isinstance(g.get(\"arguments\"), dict) else set()\n       if pk or gk:\n           inter = len(pk & gk)\n           arg_f1 += 0.0 if inter == 0 else 2*inter\/(len(pk)+len(gk))\n   n = max(1, len(probes))\n   print(f\"    [{tag}] parseable {parsed}\/{n} | tool-name acc {name_hit\/n:.3f} | arg-key F1 {arg_f1\/n:.3f}\")\n   return dict(parsed=parsed\/n, name_acc=name_hit\/n, arg_f1=arg_f1\/n)\nPROBES = build_probes(EVAL_TRAJ, CFG[\"N_EVAL_PROBES\"])\nprint(f\"    built {len(PROBES)} teacher-forced probes\")\nresults = {}\nif CFG[\"RUN_TRAINING\"]:\n   from peft import LoraConfig, get_peft_model\n   dtype = torch.bfloat16 if BF16 else torch.float32\n   model = AutoModelForCausalLM.from_pretrained(\n       CFG[\"MODEL_ID\"], torch_dtype=dtype, attn_implementation=\"sdpa\").to(DEV)\n   model.config.use_cache = False\n   model.gradient_checkpointing_enable()\n   model.enable_input_require_grads()\n   if CFG[\"RUN_EVAL\"] and PROBES and DEV == \"cuda\":\n       print(\"n[8] baseline eval\")\n       results[\"before\"] = eval_tool_calls(model, PROBES, \"base\")\n   model = get_peft_model(model, LoraConfig(\n       r=CFG[\"LORA_R\"], lora_alpha=2*CFG[\"LORA_R\"], lora_dropout=0.05,\n       bias=\"none\", task_type=\"CAUSAL_LM\",\n   model.print_trainable_parameters()\n   opt   = torch.optim.AdamW([p for p in model.parameters() if p.requires_grad],\n                             lr=CFG[\"LR\"], weight_decay=0.0, betas=(0.9, 0.95))\n   sched = get_cosine_schedule_with_warmup(opt, 5, CFG[\"MAX_STEPS\"])\n   scaler = torch.amp.GradScaler(\"cuda\", enabled=(DEV == \"cuda\" and not BF16))\n   amp_dt = torch.bfloat16 if BF16 else torch.float16\n   print(f\"n[7] training {CFG['MAX_STEPS']} steps \"\n         f\"(bs1 x accum{CFG['GRAD_ACCUM']} = {CFG['GRAD_ACCUM']} traj\/step)\")\n   model.train(); step = 0; run = None; it = iter(loader)\n   while step < CFG[\"MAX_STEPS\"]:\n       opt.zero_grad(set_to_none=True); acc = 0.0\n       for _ in range(CFG[\"GRAD_ACCUM\"]):\n           try:    batch = next(it)\n           except StopIteration:\n               it = iter(loader); batch = next(it)\n           batch = {k: v.to(DEV) for k, v in batch.items()}\n           with torch.autocast(DEV, dtype=amp_dt, enabled=(DEV == \"cuda\")):\n               loss = model(**batch).loss \/ CFG[\"GRAD_ACCUM\"]\n           scaler.scale(loss).backward() if scaler.is_enabled() else loss.backward()\n           acc += loss.item()\n       if scaler.is_enabled():\n           scaler.unscale_(opt)\n       (scaler.step(opt), scaler.update()) if scaler.is_enabled() else opt.step()\n       sched.step(); step += 1\n       run = acc if run is None else 0.9*run + 0.1*acc\n       if step % 5 == 0 or step == 1:\n           print(f\"    step {step:>3}\/{CFG['MAX_STEPS']}  loss {acc:.4f}  ema {run:.4f}  \"\n                 f\"lr {sched.get_last_lr()[0]:.2e}  ppl {math.exp(min(20, acc)):.1f}\")\n   model.save_pretrained(f\"{CFG['OUT_DIR']}\/lora_adapter\"); tok.save_pretrained(f\"{CFG['OUT_DIR']}\/lora_adapter\")\n   print(f\"    adapter -> {CFG['OUT_DIR']}\/lora_adapter\")\n   if CFG[\"RUN_EVAL\"] and PROBES and DEV == \"cuda\":\n       print(\"n[8] post-training eval\")\n       model.config.use_cache = True\n       results[\"after\"] = eval_tool_calls(model, PROBES, \"lora\")\n       model.config.use_cache = False\n   if \"before\" in results and \"after\" in results:\n       print(\"n    delta:\", {k: round(results['after'][k] - results['before'][k], 3)\n                              for k in results['after']})\n       print(\"    (30 steps on ~350 trajectories is a smoke test, not a result \u2014 \"\n             \"expect noise, and scale N_STREAM\/MAX_STEPS for anything real.)\")\n<\/code><\/pre>\n<\/div>\n<\/div>\n
We build teacher-forced evaluation probes by cutting trajectories immediately before assistant turns that contain tool calls. We load Qwen3-0.6B, measure its baseline tool-call performance, attach LoRA adapters, and fine-tune the model using gradient accumulation, mixed precision, checkpointing, clipping, and cosine learning-rate scheduling. We then evaluate the adapted model, compare its metrics with the baseline, and save the trained LoRA adapter and tokenizer.<\/p>\n
\n
\n
\n
\n
<\/div>\n
<\/div>\n
<\/div>\n<\/div>\n
Copy Code<\/span>Copied<\/span>Use a different Browser<\/span><\/a><\/div>\n<\/div>\n
struct_path = f\"{CFG['OUT_DIR']}\/aquila_{CFG['LANG']}_structured_tools.jsonl\"\nwith open(struct_path, \"w\", encoding=\"utf-8\") as f:\n   for t in TRAJ:\n       f.write(json.dumps(extract_tools(t), ensure_ascii=False) + \"n\")\nstats_path = f\"{CFG['OUT_DIR']}\/corpus_stats.json\"\nwith open(stats_path, \"w\") as f:\n   json.dump({\"n\": len(TRAJ), \"tool_freq\": dict(name_freq),\n              \"calls_mean\": stats.mean(calls_per), \"calls_max\": max(calls_per),\n              \"depth_p90\": q(depth_per, .9), \"encoded\": len(ENC),\n              \"supervised_ratio_mean\": stats.mean(sup), \"eval\": results}, f, indent=2)\nprint(f\"n[9] wrote:n    {struct_path}n    {stats_path}\")\nprint(\"done.\")\n<\/code><\/pre>\n<\/div>\n<\/div>\n
We export every parsed trajectory as a structured JSONL record containing messages, tool schemas, questions, and answers. We also save a JSON report containing corpus size, tool frequencies, trajectory statistics, supervised-token ratios, and available evaluation results. We finish the workflow with reusable dataset artifacts, analytical outputs, and model files stored in the configured output directory.<\/p>\n
In conclusion, we completed a practical pipeline for analyzing, transforming, fine-tuning, and evaluating complex tool-use trajectories from the XYZ-Aquila-SFT dataset. We preserved the original conversational structure, applied token-level supervision only to assistant responses, and used LoRA to adapt Qwen3-0.6B efficiently on a Colab-compatible GPU. We also compared baseline and post-training tool-call performance through teacher-forced evaluation and exported reusable structured records, model adapters, and analytical statistics. This workflow gives us a strong foundation for scaling tool-aware supervised fine-tuning, testing alternative sequence-length policies, and training more capable agentic language models.<\/p>\n
\n
\n<\/p>
\n<\/p>
Check out the\u00a0FULL CODES here<\/a><\/strong>.\u00a0<\/strong>Also,\u00a0feel free to follow us on\u00a0Twitter<\/mark><\/a><\/strong>\u00a0and don\u2019t forget to join our\u00a0150k+ML SubReddit<\/a><\/strong>\u00a0and Subscribe to\u00a0our Newsletter<\/a><\/strong>. Wait! are you on telegram?\u00a0now you can join us on telegram as well.<\/a><\/strong><\/p>\n
Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.?\u00a0Connect with us<\/mark><\/a><\/strong><\/p>\n
The post Fine-Tuning Tool-Calling LLMs: A Complete Guide Using XYZ-Aquila-SFT and Qwen3<\/a> appeared first on MarkTechPost<\/a>.<\/p>","protected":false},"excerpt":{"rendered":"
In this tutorial, we implement an end-to-end supervised fine-tuning pipeline for the XYZ-Aquila-SFT dataset, Hugging Face Transformers, PyTorch, and PEFT. We stream and inspect the dataset, parse multi-turn tool-use trajectories, extract structured tool calls, analyze corpus characteristics, and preserve embedded reasoning and observation patterns. We then convert tool schemas between message-embedded and structured formats, render Qwen-compatible ChatML with assistant-only loss masking, prepare a custom PyTorch dataset and collator, and fine-tune Qwen3-0.6B with LoRA. Finally, we evaluate tool-call prediction before and after training and export both the transformed dataset and corpus statistics for further experimentation. Copy CodeCopiedUse a different Browser import os, sys, subprocess CFG = dict( REPO = “XYZAILab\/XYZ-Aquila-SFT”, LANG = “en”, N_STREAM = 400, N_EVAL = 40, MODEL_ID = “Qwen\/Qwen3-0.6B”, MAX_SEQ_LEN = 2048, LENGTH_POLICY = “truncate”, RUN_TRAINING = True, MAX_STEPS = 30, GRAD_ACCUM = 8, LR = 1e-4, LORA_R = 16, RUN_EVAL = True, N_EVAL_PROBES = 24, OUT_DIR = “\/content\/aquila_out”, SEED = 0, ) os.makedirs(CFG[“OUT_DIR”], exist_ok=True) def pip(*pkgs): subprocess.run([sys.executable, “-m”, “pip”, “install”, “-q”, “-U”, *pkgs], check=False) pip(“datasets>=3.0.0”, “transformers>=4.51.0”, “peft>=0.13.0”, “accelerate>=1.0.0”) import json, re, math, random, statistics as stats from collections import Counter, defaultdict from dataclasses import dataclass, field from typing import Any, Dict, List, Optional import torch import matplotlib.pyplot as plt from datasets import load_dataset from transformers import AutoTokenizer, AutoModelForCausalLM, get_cosine_schedule_with_warmup random.seed(CFG[“SEED”]); torch.manual_seed(CFG[“SEED”]) DEV = “cuda” if torch.cuda.is_available() else “cpu” BF16 = DEV == “cuda” and torch.cuda.is_bf16_supported() print(f”device={DEV} bf16={BF16} torch={torch.__version__}”) print(f”n[1] streaming {CFG[‘REPO’]}:{CFG[‘LANG’]} …”) stream = load_dataset(CFG[“REPO”], CFG[“LANG”], split=”train”, streaming=True) RAW: List[Dict[str, Any]] = list(stream.take(CFG[“N_STREAM”])) print(f” pulled {len(RAW)} rows; keys = {list(RAW[0].keys())}”) _r = RAW[0] print(f” question[:110] : {_r[‘question’][:110]}…”) print(f” answer : {_r[‘answer’][:80]}”) print(f” number of tool calls : {_r[‘number of tool calls’]}”) print(f” trajectory len : {len(_r[‘trajectory’])} msgs”) print(f” role sequence (first8): {[m[‘role’] for m in _r[‘trajectory’][:8]]}”) We configure the dataset, model, training parameters, output directory, and reproducibility settings for the complete workflow. We install the required Hugging Face, PEFT, Accelerate, and PyTorch-related dependencies and detect whether a CUDA GPU and BF16 support are available. We then stream a limited number of XYZ-Aquila-SFT examples, inspect the dataset schema, and examine the structure of the first tool-use trajectory. Copy CodeCopiedUse a different Browser TOOLS_BLOCK_RE = re.compile(r”<tools>s*(.*?)s*<\/tools>”, re.S) THINK_RE = re.compile(r”<think>(.*?)<\/think>”, re.S) TOOL_RESP_RE = re.compile(r”<tool_response>s*(.*?)s*<\/tool_response>”, re.S) TOOLS_HDR_RE = re.compile(r”nn# Toolsnn”) def iter_json_objects(text: str, limit: int = 1): “””Nesting-safe JSON scanner. Regex like r’\\{.*?\\}’ breaks on nested `arguments` objects, which every real tool call has.””” dec, i, n, out = json.JSONDecoder(), 0, len(text), [] while i < n and len(out) < limit: while i < n and text[i] not in “{[“: i += 1 if i >= n: break try: obj, end = dec.raw_decode(text, i) except json.JSONDecodeError: i += 1 continue out.append(obj); i = end return out def parse_tool_calls(content: str) -> List[Dict[str, Any]]: calls = [] for m in re.finditer(r”<tool_call>”, content): got = iter_json_objects(content[m.end():], limit=1) if got: calls.append(got[0]) return calls @dataclass class Trajectory: question: str answer: str declared_calls: int messages: List[Dict[str, str]] system_core: str = “” tools: List[Dict[str, Any]] = field(default_factory=list) tools_suffix: str = “” calls: List[Dict[str, Any]] = field(default_factory=list) n_observations: int = 0 n_think: int = 0 @property def tool_names(self): return [c.get(“name”, “?”) for c in self.calls] @property def depth(self): return len(self.messages) def parse_row(row: Dict[str, Any]) -> Trajectory: msgs = [{“role”: m[“role”], “content”: m[“content”]} for m in row[“trajectory”]] t = Trajectory(row[“question”], row[“answer”], row[“number of tool calls”], msgs) if msgs and msgs[0][“role”] == “system”: sysmsg = msgs[0][“content”] split = TOOLS_HDR_RE.search(sysmsg) if split: t.system_core = sysmsg[:split.start()] t.tools_suffix = sysmsg[split.start():] else: t.system_core = sysmsg blk = TOOLS_BLOCK_RE.search(sysmsg) if blk: t.tools = iter_json_objects(blk.group(1), limit=64) for m in msgs: if m[“role”] == “assistant”: t.calls += parse_tool_calls(m[“content”]) t.n_think += len(THINK_RE.findall(m[“content”])) else: t.n_observations += len(TOOL_RESP_RE.findall(m[“content”])) return t TRAJ = [parse_row(r) for r in RAW] t0 = TRAJ[0] print(f”n[2] parsed {len(TRAJ)} trajectories”) print(f” tool schemas found : {[fn.get(‘function’, fn).get(‘name’) for fn in t0.tools]}”) print(f” parsed calls : {len(t0.calls)} (declared {t0.declared_calls})”) print(f” observations : {t0.n_observations} think blocks: {t0.n_think}”) if t0.calls: print(f” sample call : {json.dumps(t0.calls[0], ensure_ascii=False)[:200]}”) agree = sum(len(t.calls) == t.declared_calls for t in TRAJ) print(f” parser vs ‘number of tool calls’: {agree}\/{len(TRAJ)} exact match”) calls_per = [len(t.calls) for t in TRAJ] depth_per = [t.depth for t in TRAJ] chars_per = [sum(len(m[“content”]) for m in t.messages) for t in TRAJ] name_freq = Counter(n for t in TRAJ for n in t.tool_names) argkey_freq = defaultdict(Counter) for t in TRAJ: for c in t.calls: args = c.get(“arguments”, {}) if isinstance(args, dict): for k in args: argkey_freq[c.get(“name”, “?”)][k] += 1 def q(xs, p): xs = sorted(xs); return xs[min(len(xs) – 1, int(p * len(xs)))] print(“n[3] corpus statistics”) print(f” tool calls \/ traj : mean {stats.mean(calls_per):.1f} p50 {q(calls_per,.5)} ” f”p90 {q(calls_per,.9)} max {max(calls_per)}”) print(f” messages \/ traj : mean {stats.mean(depth_per):.1f} p90 {q(depth_per,.9)} max {max(depth_per)}”) print(f” chars \/ traj : mean {stats.mean(chars_per):,.0f} p90 {q(chars_per,.9):,}”) print(f” tool distribution : {dict(name_freq)}”) for k, v in argkey_freq.items(): print(f” {k:<24} arg keys -> {dict(v.most_common(6))}”) tot = sum(chars_per); top = sum(sorted(chars_per)[-max(1, len(chars_per)\/\/10):]) print(f” top-10% longest trajectories hold {100*top\/tot:.1f}% of all characters”) fig, ax = plt.subplots(1, 3, figsize=(15, 3.6)) ax[0].hist(calls_per, bins=40); ax[0].set_yscale(“log”); ax[0].set_title(“tool calls \/ trajectory”) ax[1].hist(depth_per, bins=40); ax[1].set_yscale(“log”); ax[1].set_title(“messages \/ trajectory”) ax[2].bar(list(name_freq), list(name_freq.values())); ax[2].set_title(“tool usage”); ax[2].tick_params(axis=”x”, rotation=20) plt.tight_layout(); plt.show() We define nesting-safe utilities for extracting JSON tool calls, reasoning blocks, observations, and embedded tool schemas from each conversation. We convert every raw dataset row into a structured trajectory object and verify that the parsed tool-call counts match the values declared by the dataset. We then calculate corpus-level statistics and visualize the distributions of tool calls, message depth, trajectory size, and tool usage frequency. Copy CodeCopiedUse a different Browser QWEN3_TOOLS_TMPL = ( “You are provided with function signatures within <tools><\/tools> XML tags:n<tools>n” “{lines}n<\/tools>nnFor each function call, return a json object with function name ” “and arguments within <tool_call><\/tool_call> XML tags:n<tool_call>n” ‘{{“name”: <function-name>, “arguments”: <args-json-object>}}n<\/tool_call>’ ) def extract_tools(t: Trajectory) -> Dict[str, Any]: “””message-embedded schemas -> {‘messages’: […], ‘tools’: […]}””” msgs = [dict(m) for m in t.messages] if msgs and msgs[0][“role”] == “system”: msgs[0][“content”] = t.system_core return {“messages”: msgs, “tools”: t.tools, “question”: t.question, “answer”: t.answer} def render_tools(rec: Dict[str, Any]) -> List[Dict[str, str]]: “””inverse: structured tools -> schemas re-embedded in the system message”””<\/p>","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"pmpro_default_level":"","site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"_pvb_checkbox_block_on_post":false,"footnotes":""},"categories":[52,5,7,1],"tags":[],"class_list":["post-111606","post","type-post","status-publish","format-standard","hentry","category-ai-club","category-committee","category-news","category-uncategorized","pmpro-has-access"],"acf":[],"yoast_head":"\nFine-Tuning Tool-Calling LLMs: A Complete Guide Using XYZ-Aquila-SFT and Qwen3 - YouZum<\/title>\n<meta name=\"description\" content=\"\u0e01\u0e34\u0e08\u0e01\u0e23\u0e23\u0e21\u0e40\u0e01\u0e35\u0e48\u0e22\u0e27\u0e01\u0e31\u0e1a\u0e42\u0e14\u0e23\u0e19\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/youzum.net\/es\/fine-tuning-tool-calling-llms-a-complete-guide-using-xyz-aquila-sft-and-qwen3\/\" \/>\n<meta property=\"og:locale\" content=\"es_ES\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Fine-Tuning Tool-Calling LLMs: A Complete Guide Using XYZ-Aquila-SFT and Qwen3 - YouZum\" \/>\n<meta property=\"og:description\" content=\"\u0e01\u0e34\u0e08\u0e01\u0e23\u0e23\u0e21\u0e40\u0e01\u0e35\u0e48\u0e22\u0e27\u0e01\u0e31\u0e1a\u0e42\u0e14\u0e23\u0e19\" \/>\n<meta property=\"og:url\" content=\"https:\/\/youzum.net\/es\/fine-tuning-tool-calling-llms-a-complete-guide-using-xyz-aquila-sft-and-qwen3\/\" \/>\n<meta property=\"og:site_name\" content=\"YouZum\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/DroneAssociationTH\/\" \/>\n<meta property=\"article:published_time\" content=\"2026-08-15T21:49:56+00:00\" \/>\n<meta name=\"author\" content=\"admin NU\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Escrito por\" \/>\n\t<meta name=\"twitter:data1\" content=\"admin NU\" \/>\n\t<meta name=\"twitter:label2\" content=\"Tiempo de lectura\" \/>\n\t<meta name=\"twitter:data2\" content=\"16 minutos\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/youzum.net\/fine-tuning-tool-calling-llms-a-complete-guide-using-xyz-aquila-sft-and-qwen3\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/youzum.net\/fine-tuning-tool-calling-llms-a-complete-guide-using-xyz-aquila-sft-and-qwen3\/\"},\"author\":{\"name\":\"admin NU\",\"@id\":\"https:\/\/yousum.gpucore.co\/#\/schema\/person\/97fa48242daf3908e4d9a5f26f4a059c\"},\"headline\":\"Fine-Tuning Tool-Calling LLMs: A Complete Guide Using XYZ-Aquila-SFT and Qwen3\",\"datePublished\":\"2026-08-15T21:49:56+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/youzum.net\/fine-tuning-tool-calling-llms-a-complete-guide-using-xyz-aquila-sft-and-qwen3\/\"},\"wordCount\":620,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/yousum.gpucore.co\/#organization\"},\"articleSection\":[\"AI\",\"Committee\",\"News\",\"Uncategorized\"],\"inLanguage\":\"es\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/youzum.net\/fine-tuning-tool-calling-llms-a-complete-guide-using-xyz-aquila-sft-and-qwen3\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/youzum.net\/fine-tuning-tool-calling-llms-a-complete-guide-using-xyz-aquila-sft-and-qwen3\/\",\"url\":\"https:\/\/youzum.net\/fine-tuning-tool-calling-llms-a-complete-guide-using-xyz-aquila-sft-and-qwen3\/\",\"name\":\"Fine-Tuning Tool-Calling LLMs: A Complete Guide Using XYZ-Aquila-SFT and Qwen3 - YouZum\",\"isPartOf\":{\"@id\":\"https:\/\/yousum.gpucore.co\/#website\"},\"datePublished\":\"2026-08-15T21:49:56+00:00\",\"description\":\"\u0e01\u0e34\u0e08\u0e01\u0e23\u0e23\u0e21\u0e40\u0e01\u0e35\u0e48\u0e22\u0e27\u0e01\u0e31\u0e1a\u0e42\u0e14\u0e23\u0e19\",\"breadcrumb\":{\"@id\":\"https:\/\/youzum.net\/fine-tuning-tool-calling-llms-a-complete-guide-using-xyz-aquila-sft-and-qwen3\/#breadcrumb\"},\"inLanguage\":\"es\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/youzum.net\/fine-tuning-tool-calling-llms-a-complete-guide-using-xyz-aquila-sft-and-qwen3\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/youzum.net\/fine-tuning-tool-calling-llms-a-complete-guide-using-xyz-aquila-sft-and-qwen3\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/youzum.net\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Fine-Tuning Tool-Calling LLMs: A Complete Guide Using XYZ-Aquila-SFT and Qwen3\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/yousum.gpucore.co\/#website\",\"url\":\"https:\/\/yousum.gpucore.co\/\",\"name\":\"YouSum\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/yousum.gpucore.co\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/yousum.gpucore.co\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"es\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/yousum.gpucore.co\/#organization\",\"name\":\"Drone Association Thailand\",\"url\":\"https:\/\/yousum.gpucore.co\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"es\",\"@id\":\"https:\/\/yousum.gpucore.co\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/youzum.net\/wp-content\/uploads\/2024\/11\/tranparent-logo.png\",\"contentUrl\":\"https:\/\/youzum.net\/wp-content\/uploads\/2024\/11\/tranparent-logo.png\",\"width\":300,\"height\":300,\"caption\":\"Drone Association Thailand\"},\"image\":{\"@id\":\"https:\/\/yousum.gpucore.co\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/DroneAssociationTH\/\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/yousum.gpucore.co\/#\/schema\/person\/97fa48242daf3908e4d9a5f26f4a059c\",\"name\":\"admin NU\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"es\",\"@id\":\"https:\/\/yousum.gpucore.co\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/youzum.net\/wp-content\/uploads\/avatars\/2\/1746849356-bpfull.png\",\"contentUrl\":\"https:\/\/youzum.net\/wp-content\/uploads\/avatars\/2\/1746849356-bpfull.png\",\"caption\":\"admin NU\"},\"url\":\"https:\/\/youzum.net\/es\/members\/adminnu\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Fine-Tuning Tool-Calling LLMs: A Complete Guide Using XYZ-Aquila-SFT and Qwen3 - YouZum","description":"\u0e01\u0e34\u0e08\u0e01\u0e23\u0e23\u0e21\u0e40\u0e01\u0e35\u0e48\u0e22\u0e27\u0e01\u0e31\u0e1a\u0e42\u0e14\u0e23\u0e19","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/youzum.net\/es\/fine-tuning-tool-calling-llms-a-complete-guide-using-xyz-aquila-sft-and-qwen3\/","og_locale":"es_ES","og_type":"article","og_title":"Fine-Tuning Tool-Calling LLMs: A Complete Guide Using XYZ-Aquila-SFT and Qwen3 - YouZum","og_description":"\u0e01\u0e34\u0e08\u0e01\u0e23\u0e23\u0e21\u0e40\u0e01\u0e35\u0e48\u0e22\u0e27\u0e01\u0e31\u0e1a\u0e42\u0e14\u0e23\u0e19","og_url":"https:\/\/youzum.net\/es\/fine-tuning-tool-calling-llms-a-complete-guide-using-xyz-aquila-sft-and-qwen3\/","og_site_name":"YouZum","article_publisher":"https:\/\/www.facebook.com\/DroneAssociationTH\/","article_published_time":"2026-08-15T21:49:56+00:00","author":"admin NU","twitter_card":"summary_large_image","twitter_misc":{"Escrito por":"admin NU","Tiempo de lectura":"16 minutos"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/youzum.net\/fine-tuning-tool-calling-llms-a-complete-guide-using-xyz-aquila-sft-and-qwen3\/#article","isPartOf":{"@id":"https:\/\/youzum.net\/fine-tuning-tool-calling-llms-a-complete-guide-using-xyz-aquila-sft-and-qwen3\/"},"author":{"name":"admin NU","@id":"https:\/\/yousum.gpucore.co\/#\/schema\/person\/97fa48242daf3908e4d9a5f26f4a059c"},"headline":"Fine-Tuning Tool-Calling LLMs: A Complete Guide Using XYZ-Aquila-SFT and Qwen3","datePublished":"2026-08-15T21:49:56+00:00","mainEntityOfPage":{"@id":"https:\/\/youzum.net\/fine-tuning-tool-calling-llms-a-complete-guide-using-xyz-aquila-sft-and-qwen3\/"},"wordCount":620,"commentCount":0,"publisher":{"@id":"https:\/\/yousum.gpucore.co\/#organization"},"articleSection":["AI","Committee","News","Uncategorized"],"inLanguage":"es","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/youzum.net\/fine-tuning-tool-calling-llms-a-complete-guide-using-xyz-aquila-sft-and-qwen3\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/youzum.net\/fine-tuning-tool-calling-llms-a-complete-guide-using-xyz-aquila-sft-and-qwen3\/","url":"https:\/\/youzum.net\/fine-tuning-tool-calling-llms-a-complete-guide-using-xyz-aquila-sft-and-qwen3\/","name":"Fine-Tuning Tool-Calling LLMs: A Complete Guide Using XYZ-Aquila-SFT and Qwen3 - YouZum","isPartOf":{"@id":"https:\/\/yousum.gpucore.co\/#website"},"datePublished":"2026-08-15T21:49:56+00:00","description":"\u0e01\u0e34\u0e08\u0e01\u0e23\u0e23\u0e21\u0e40\u0e01\u0e35\u0e48\u0e22\u0e27\u0e01\u0e31\u0e1a\u0e42\u0e14\u0e23\u0e19","breadcrumb":{"@id":"https:\/\/youzum.net\/fine-tuning-tool-calling-llms-a-complete-guide-using-xyz-aquila-sft-and-qwen3\/#breadcrumb"},"inLanguage":"es","potentialAction":[{"@type":"ReadAction","target":["https:\/\/youzum.net\/fine-tuning-tool-calling-llms-a-complete-guide-using-xyz-aquila-sft-and-qwen3\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/youzum.net\/fine-tuning-tool-calling-llms-a-complete-guide-using-xyz-aquila-sft-and-qwen3\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/youzum.net\/"},{"@type":"ListItem","position":2,"name":"Fine-Tuning Tool-Calling LLMs: A Complete Guide Using XYZ-Aquila-SFT and Qwen3"}]},{"@type":"WebSite","@id":"https:\/\/yousum.gpucore.co\/#website","url":"https:\/\/yousum.gpucore.co\/","name":"YouSum","description":"","publisher":{"@id":"https:\/\/yousum.gpucore.co\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/yousum.gpucore.co\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"es"},{"@type":"Organization","@id":"https:\/\/yousum.gpucore.co\/#organization","name":"Drone Association Thailand","url":"https:\/\/yousum.gpucore.co\/","logo":{"@type":"ImageObject","inLanguage":"es","@id":"https:\/\/yousum.gpucore.co\/#\/schema\/logo\/image\/","url":"https:\/\/youzum.net\/wp-content\/uploads\/2024\/11\/tranparent-logo.png","contentUrl":"https:\/\/youzum.net\/wp-content\/uploads\/2024\/11\/tranparent-logo.png","width":300,"height":300,"caption":"Drone Association Thailand"},"image":{"@id":"https:\/\/yousum.gpucore.co\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/DroneAssociationTH\/"]},{"@type":"Person","@id":"https:\/\/yousum.gpucore.co\/#\/schema\/person\/97fa48242daf3908e4d9a5f26f4a059c","name":"admin NU","image":{"@type":"ImageObject","inLanguage":"es","@id":"https:\/\/yousum.gpucore.co\/#\/schema\/person\/image\/","url":"https:\/\/youzum.net\/wp-content\/uploads\/avatars\/2\/1746849356-bpfull.png","contentUrl":"https:\/\/youzum.net\/wp-content\/uploads\/avatars\/2\/1746849356-bpfull.png","caption":"admin NU"},"url":"https:\/\/youzum.net\/es\/members\/adminnu\/"}]}},"rttpg_featured_image_url":null,"rttpg_author":{"display_name":"admin NU","author_link":"https:\/\/youzum.net\/es\/members\/adminnu\/"},"rttpg_comment":0,"rttpg_category":"<a href=\"https:\/\/youzum.net\/es\/category\/ai-club\/\" rel=\"category tag\">AI<\/a> <a href=\"https:\/\/youzum.net\/es\/category\/committee\/\" rel=\"category tag\">Committee<\/a> <a href=\"https:\/\/youzum.net\/es\/category\/news\/\" rel=\"category tag\">News<\/a> <a href=\"https:\/\/youzum.net\/es\/category\/uncategorized\/\" rel=\"category tag\">Uncategorized<\/a>","rttpg_excerpt":"In this tutorial, we implement an end-to-end supervised fine-tuning pipeline for the XYZ-Aquila-SFT dataset, Hugging Face Transformers, PyTorch, and PEFT. We stream and inspect the dataset, parse multi-turn tool-use trajectories, extract structured tool calls, analyze corpus characteristics, and preserve embedded reasoning and observation patterns. We then convert tool schemas between message-embedded and structured formats, render…","_links":{"self":[{"href":"https:\/\/youzum.net\/es\/wp-json\/wp\/v2\/posts\/111606","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/youzum.net\/es\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/youzum.net\/es\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/youzum.net\/es\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/youzum.net\/es\/wp-json\/wp\/v2\/comments?post=111606"}],"version-history":[{"count":0,"href":"https:\/\/youzum.net\/es\/wp-json\/wp\/v2\/posts\/111606\/revisions"}],"wp:attachment":[{"href":"https:\/\/youzum.net\/es\/wp-json\/wp\/v2\/media?parent=111606"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/youzum.net\/es\/wp-json\/wp\/v2\/categories?post=111606"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/youzum.net\/es\/wp-json\/wp\/v2\/tags?post=111606"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}