We will use Grok 3.5 (maybe we should call it 4), which has advanced reasoning, to rewrite the entire corpus of human knowledge, adding missing information and deleting errors.

Then retrain on that.

Far too much garbage in any foundation model trained on uncorrected data.

Source.

More Context

Source.

Source.

  • brucethemoose@lemmy.worldEnglish
    3·
    20 hours ago

    It’s not so simple, there are papers on zero data ‘self play’ or other schemes for using other LLM’s output.

    Distillation is probably the only one you’d want for a pretrain, specifically.