Model Cards

When someone picks up an AI model, whether a data scientist reusing it, a product team building on it or a customer evaluating a vendor, they need to know what it is for, how well it works and where it might fail. Too often, that information is scattered across code, notebooks and the memories of the people who built it. Model cards collect it in one place. Like a nutrition label for a model, a model card gives a concise, standardized summary of its purpose, performance, limits and risks, written so that people outside the development team can understand it. The goal is honest, useful information, not marketing.

The idea was introduced in a 2019 research paper by Margaret Mitchell and colleagues, titled Model Cards for Model Reporting. It has since become common practice. Major AI developers publish model cards or system cards for their models, open model repositories encourage them, and regulations and standards such as the EU AI Act and ISO 42001 require technical documentation that model cards help provide. A related idea, datasheets for datasets, documents the data used to train models. For data science, risk, compliance and product teams, model cards are a simple, effective governance tool.

Definition of Model Cards

A model card is a short document accompanying a trained machine learning model that provides key information about its development, intended uses and out-of-scope uses, performance across relevant conditions and groups, training and evaluation data, limitations, risks and ethical considerations.

Model cards support AI transparency, provide documentation for AI model governance and record the results of testing for algorithmic bias.

What a Model Card Contains

Model details include the developer, version, model type, date and contact information. Version information ensures readers know exactly which model the card describes.

Intended use describes the purposes and users the model was designed for, and out-of-scope uses describe what it should not be used for. Clear out-of-scope uses are often the most valuable part of the card.

Factors and metrics describe the conditions, groups and measures used to evaluate the model. Choosing the right factors shows readers where performance may differ.

Evaluation results report performance overall and broken down by relevant groups and conditions. Breakdowns reveal weaknesses that overall averages can hide.

Training and evaluation data describe the datasets used, including sources, size and known gaps. Linking to datasheets gives readers more detail when they need it.

Limitations, risks and ethical considerations describe known weaknesses, potential harms and recommended safeguards. Being candid about limitations builds trust with readers.

Related Documentation

Datasheets for datasets document the motivation, composition, collection process and recommended uses of a dataset. They were proposed by Timnit Gebru and colleagues in research first published in 2018.

System cards describe an AI system as a whole, including multiple models, safeguards and how they work together, and are often published for large generative AI systems.

Technical documentation required by regulations is usually more detailed than a model card, but model cards can form a useful summary within it.

Together, these documents create a record of how AI systems were built and how they should be used.

Importance of Model Cards in AI Governance

Model cards prevent misuse. Stating intended and out-of-scope uses helps stop models from being applied where they will perform poorly or cause harm. This is especially important when models are shared across teams.

They make risks visible. Performance broken down by group and stated limitations help reviewers, users and customers judge whether a model is suitable. Decisions about use become informed rather than assumed.

They also support compliance and accountability. Model cards provide documentation that regulators, auditors and customers increasingly expect. They also help new team members understand models built before they arrived.

Model Cards and Financial Management

Clear documentation speeds up reuse of existing models, avoiding the cost of building new ones unnecessarily.

Model cards reduce the time needed for reviews, audits and customer due diligence. Reviewers can start from a clear summary rather than piecing information together.

Documenting limitations up front can prevent costly failures when models are used outside their intended scope.

Templates and automation keep the cost of producing model cards low. Some fields can be filled automatically from training and evaluation pipelines.

Common Challenges

Model cards can be written once and never updated, so they no longer match the model in production. Linking card updates to model releases keeps them current.

Teams may fill in templates superficially, leaving out the limitations and group results that matter most. Reviewing cards during approval helps maintain quality.

Vendors may provide limited model cards, leaving gaps for organizations that rely on their models. Organizations can fill some gaps with their own testing.