01
Frame a valid modeling task
Define the prediction unit, target, observation time, prediction time, population, decision, error costs, baseline, and success metric.
From question to evaluated model
CategoryData & Research
Codex Skills for Data Science
Codex Skills for Data Science give Codex structured instructions for framing prediction problems, exploring data, preventing leakage, building preprocessing pipelines, comparing models, evaluating errors, documenting experiments, and preparing reproducible outputs. Download the skills for projects where a model must answer a real question, not just produce a score.
Problem framingFeature pipelinesModel evaluationExperiment tracking
01
02
03
04
05
06
07
08
09
02
03
04
05
06
07
08
09
task: predict customer retention
inspect:
- target definition
- split strategy
- leakage risk
verify: baseline + metrics + error analysisA high score can answer the wrong question
Data science succeeds when the target, evaluation split, metric, and deployment context all reflect the decision that matters.
Modeling mistakes often begin before training. A vague target, future information in features, an unrepresentative split, class imbalance, or a convenient metric can make results look stronger than they are.
These skills guide Codex through exploratory data analysis, feature engineering, machine learning pipelines, cross-validation, model comparison, interpretation, and reproducible reporting with assumptions kept visible.
What the downloadable skills can do
What Codex Skills for Data Science can help investigate
02
Build leakage-safe pipelines
Split data appropriately, fit preprocessing only on training data, handle missing values, encode features, and keep transformations reproducible.
03
Compare models fairly
Establish simple baselines, use suitable cross-validation, tune within the validation process, report multiple relevant metrics, and retain the untouched test set.
04
Explain errors and limitations
Analyze performance by segment, inspect false positives and negatives, test sensitivity, document drift risks, and state where the model should not be used.
An experiment with a defensible test
Lock the question and evaluation design before optimizing the model
The process prevents test-set knowledge from leaking into feature choices, preprocessing, or tuning.
workflow.statusREADY
Context → Plan → Work → Verify
01
Translate the decision into a target
Define who or what is predicted, at what time, over which horizon, using which available information, and why the output is useful.
02
Audit data and design splits
Check quality, missingness, duplicates, temporal order, groups, imbalance, sensitive attributes, leakage, and representative train-validation-test separation.
03
Build baselines and pipelines
Start with a simple reference, create repeatable preprocessing, compare justified algorithms, tune carefully, and track every experiment.
04
Evaluate beyond one metric
Review calibration, thresholds, segment performance, stability, error costs, interpretability, limitations, and reproducibility before recommending use.
Data science across project stages
Use the right workflow for exploration, experimentation, or delivery
A notebook exploration and a production prediction service need different levels of control.
Exploratory modeling
Understand the dataset, test feasibility, identify signal, establish baselines, visualize errors, and define the next experiment.
Machine learning experiments
Build pipelines, compare estimators, cross-validate, tune, track results, and preserve an honest final evaluation.
NLP and unstructured data
Prepare text or other unstructured inputs, define representations, manage splits, evaluate tasks, and inspect qualitative errors.
Production preparation
Package preprocessing and inference together, define input contracts, test serialization, plan monitoring, and document retraining triggers.
Common modeling work
Use the skills from dataset audit to model report
The workflow supports classification, regression, clustering, ranking, forecasting, and other tasks when their assumptions are stated.
01Define a prediction target
02Audit features for leakage
03Create train and test splits
04Build preprocessing pipelines
05Establish a baseline model
06Compare models with cross-validation
07Analyze errors by segment
08Write a reproducible model report
People responsible for model evidence
Who Is This For?
These skills are for practitioners who need results that can be reproduced, challenged, and connected to a real decision.
Data scientists
Structure experiments, prevent leakage, compare models fairly, interpret errors, and document limitations.
Machine learning engineers
Turn validated experiments into testable pipelines, model artifacts, input contracts, and monitored inference systems.
Researchers
Keep data preparation, hypotheses, evaluation design, metrics, experiments, and conclusions transparent.
Product and analytics teams
Clarify whether prediction is useful, choose business-relevant metrics, understand error costs, and evaluate deployment readiness.
Good to know:
The skills cannot make biased or unrepresentative data suitable, and model performance does not establish causation. High-impact uses require domain review, fairness and privacy assessment, security, monitoring, and qualified approval.
Set up a reproducible experiment repository
Installation Guide
Install the skills where Codex can access data documentation, notebooks, pipeline code, environments, experiments, and reporting rules.
Download and extract the data science skills
Keep SKILL.md with its framing, leakage, experiment, evaluation, and reporting guidance.
Choose project or personal scope
Use project scope for one dataset and target, or personal scope for a reusable modeling process.
Document the prediction setting
Record population, target, observation window, prediction time, horizon, decision, error costs, sensitive attributes, and prohibited uses.
Declare data and tool constraints
Add schemas, source dates, access rules, approved libraries, compute limits, random seed policy, tracking method, and output formats.
Start with an evaluation plan
Define the split strategy, baseline, metrics, segments, final test procedure, and acceptance criteria before model selection.
Before training a model
Frequently Asked Questions
Answers about algorithms, leakage, cross-validation, notebooks, and production use.
Can the skills choose the best algorithm?+
They can compare justified candidates against a baseline using appropriate validation. The best choice also depends on error costs, interpretability, latency, maintenance, and deployment constraints.
How do they prevent data leakage?+
The workflow fixes observation and prediction times, audits feature availability, separates data before learned preprocessing, and uses pipelines within cross-validation.
Do they support deep learning?+
They can support deep learning when the environment and task justify it, but still require baselines, valid splits, controlled experiments, error analysis, and reproducibility.
Can they turn a notebook into production code?+
They can separate exploration from reusable preprocessing, training, evaluation, serialization, and inference modules, then add tests and input contracts.
Will the model be safe to deploy?+
Not automatically. Deployment needs data and concept drift monitoring, privacy and fairness review, security, rollback, ownership, and domain-specific validation.
Make the evaluation harder to fool
Give Codex a data science workflow that protects the test, explains the errors, and connects the model to a real decision.
Defined target. Leakage-safe pipeline. Honest comparison. Limits stated before deployment.