What are weights and biases?
In a neural network, each connection has a weight that scales a signal, and each unit has a bias that shifts it. Training starts with random values and adjusts every one to reduce error, using a loss function and gradient descent. After training the parameters are frozen, and they are the model. Sharing a model means sharing these numbers, which is what an open-weight release is.
A tiny worked example: a layer with 3 inputs and 2 outputs has 3 x 2 = 6 weights plus 2 biases, so 8 parameters. Count it for every layer and a network with millions or billions of them results.
How do parameters differ from hyperparameters and settings?
| Kind | Who sets it | When | Examples |
|---|---|---|---|
| Parameters | Training, automatically | During training | Weights, biases |
| Hyperparameters | You | Before training | Learning rate, batch size, number of layers |
| Request settings | You | Each call | Temperature, maximum output length |
People searching “model parameters” often mean the third row: the dials on an API call. Those change how a trained model samples its answer and do not change what it learned.
What does parameter count tell you?
Parameter count is a rough measure of capacity and cost. More parameters can store more patterns, and they take more memory and compute to run: a rule of thumb is parameters times bytes per number gives the memory for the weights alone, so a 7 billion parameter model at 2 bytes each needs about 14 GB before anything else. Count is a poor measure of quality. A smaller model trained well on good data can beat a bigger one, and mixture-of-experts models use only a share of their parameters for each token, so total and active counts differ.
How do you shrink a model’s parameters?
Quantization stores each parameter in fewer bits, cutting memory with a small quality loss. Model distillation trains a smaller model to imitate a larger one. A small language model starts with fewer parameters by design. Choose the smallest model that passes your own evaluation, because every extra parameter is paid for on every request.