Examples include model developers such as OpenAI and cloud platforms such as Amazon Bedrock.
What does an LLM provider manage?
The provider typically handles model hosting, inference, API access, capacity, and service updates. The customer still owns the application logic, permissions, evaluation, and handling of model failures. Provider features may also include fine-tuning, tools, safety controls, and observability.
A model API is a dependency inside a system, not the system itself.
How should a company choose a provider?
Start with representative tasks and explicit constraints. Compare task success, latency, throughput, data terms, regions, reliability, support, and total cost. Public benchmarks help narrow candidates but cannot replace application tests. Review AI vendor lock-in and model portability before adopting provider-specific features throughout the codebase.
What belongs in an LLM provider scorecard?
A useful scorecard separates model capability from service quality. It records task success, structured output reliability, supported regions, data controls, rate limits, latency, availability, support, and accepted-output cost. Requirements should be weighted before results are known so a famous provider does not win by default. Every provider must be tested with the same representative inputs and acceptance rules. Keep the evidence in an AI evaluation harness and revisit the scorecard when a model version, contract, or production workload changes materially.
Hugging Face’s inference providers documentation shows how one platform distinguishes models from the services that run them.