Concepts -- 2.7 Why scaling works

← Concepts

GPT: So I think your explanation of scaling is actually cleaner:

• NN, CNN, and TF are pattern classifiers.
• Larger models have more weights.
• More weights allow more complex classification boundaries.
• Therefore larger models can classify patterns more accurately.

For example, for the Tiny TF demo, the new token is for all practical purposes the classifier of the entire input. In this case only 8 tokens. Scaling increases the number and complexity of patterns that the model can distinguish and classify.

26.0624 (v1 26.0624)