convert(fp8->int4): --mtp/--indexer respected on the --indir path, no more container overwrite (#355)

The local (`--indir`) branch of main() ignored --mtp and --indexer entirely:
it always called convert_shard() without keep_mtp/keep_idx and always wrote
`out-NNNNN.safetensors`. So the documented two-pass workflow — convert the main
model into an outdir, then run `--mtp` into the SAME outdir for the head — did
the opposite on the local path: with --mtp defaulting ebits to 8 and keep_mtp
staying False, it silently re-converted the whole model to per-row int8 and
overwrote the finished fmt=4 container shard by shard, printing nothing wrong.
@mohamedmastouri2000-boop lost 137 of 141 freshly-converted g64 shards to this
while building the public container for #298/#326.

The --indir branch now mirrors the download path: keep_mtp=a.mtp /
keep_idx=a.indexer passed through, output named out-mtp-/out-idx-/out- by mode,
empty shards skipped (an MTP pass emits only shards containing layer n_layers),
and config/tokenizer copied only on the main pass (the head/idx passes land in
an already-complete outdir).

Verified with a synthetic 2-layer + MTP-shard model: after the main pass, a
second --mtp pass into the same outdir leaves out-00000's md5 BYTE-IDENTICAL
and writes out-mtp-00000 alongside it. The bug would have changed that md5.

Reported-by: mohamedmastouri2000-boop <#355>

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

c/tools/convert_fp8_to_int4.py

@@ -379,14 +379,32 @@ def main():
     if a.indir:    # conversione locale (test)
         shards = sorted(glob.glob(os.path.join(a.indir, "*.safetensors")))
         from safetensors.numpy import save_file
+        # BUG #355: questo ramo ignorava --mtp/--indexer. Con --mtp scriveva
+        # out-NNNNN (gli STESSI nomi di una conversione normale) in ebits=8 e
+        # keep_mtp=False -> il "secondo passaggio MTP" nella stessa outdir
+        # SOVRASCRIVEVA il container gia' finito con una riconversione int8
+        # completa, in silenzio (137/141 shard distrutti prima di accorgersene).
+        # Ora il ramo locale rispecchia il download path: prefisso corretto,
+        # flag passate, shard vuoti saltati.
+        prefix = "out-mtp-" if a.mtp else "out-idx-" if a.indexer else "out-"
+        n = 0
         for i, sp in enumerate(shards):
-            out = {}; convert_shard(sp, out, a.n_layers, a.ebits, a.io_bits, a.xbits, group_size=a.group_size, bits_map=bits_map)
+            out = {}
-            save_file(out, os.path.join(a.outdir, f"out-{i:05d}.safetensors"))
+            convert_shard(sp, out, a.n_layers, a.ebits, a.io_bits, a.xbits,
-        # copia config + tokenizer
+                          keep_mtp=a.mtp, keep_idx=a.indexer,
-        for fn in ["config.json"]:
+                          group_size=a.group_size, bits_map=bits_map)
-            src = os.path.join(a.indir, fn)
+            if not out:                                   # shard senza MTP/idx: niente file (come il download path)
-            if os.path.exists(src): shutil.copy(src, a.outdir)
+                continue
-        print(f"converted {len(shards)} shards -> {a.outdir}")
+            save_file(out, os.path.join(a.outdir, f"{prefix}{n:05d}.safetensors"))
+            n += 1
+        # config/tokenizer solo per la conversione principale — i passaggi mtp/idx
+        # vanno nella stessa outdir di un container gia' completo di metadati.
+        if not a.mtp and not a.indexer:
+            for fn in ["config.json"]:
+                src = os.path.join(a.indir, fn)
+                if os.path.exists(src): shutil.copy(src, a.outdir)
+        tag = "MTP" if a.mtp else "indexer" if a.indexer else "main"
+        print(f"converted {n} {tag} shard(s) -> {a.outdir} ({prefix}NNNNN)")
         return
 
     # reale: scarica shard per shard, converte, cancella