MahdiKordian/DataForge

Load a dataset, see what is wrong with it, fix it step by step, and export something your model can use.
Automated data preparation for machine learning, with no pandas code required.

Features · Quick start · Usage guide · FAQ · Contributing

• About
• Features
• Quick start
• Usage guide
• Supported inputs and outputs
• Sample datasets
• How it works
• Project structure
• Tech stack
• FAQ
• Roadmap
• Contributing
• License
• Acknowledgements
• Author

Most of a machine learning project is not modeling. It is filling gaps, removing duplicates, taming outliers, encoding categories, scaling columns and splitting without leaking data, repeated in notebook after notebook.

DataForge turns that routine into one guided web app. It inspects your dataset, flags what is wrong, applies the fixes you choose, records every step in a log, and hands you a clean dataset and a report at the end. It is built for students, analysts and ML engineers who want reliable preprocessing without rewriting the same pandas code every time.

• Flexible loading. Upload a file, paste from a spreadsheet, load from a URL, or type a small table. Separators, encodings and Excel sheets are detected automatically or set by hand.
• Health dashboard. An ML readiness score and a list of detected issues ranked by severity, so you know what to fix first.
• Guided preparation. Five steps (Clean, Transform, Encode, Scale, Split) with sensible defaults and full control over every option.
• Deep analysis. Nine views covering distributions, missing values, correlations, outliers, categoricals, the target, PCA and before vs after.
• Undo and reset. Every step is snapshotted, so you can go back or start over without reloading.
• Full audit trail. Each change is logged and can be downloaded, so your preparation is documented and repeatable.
• Ready-to-share output. Export the prepared data and a standalone HTML or Markdown report.
• Private by design. Your data lives only in the current session and is never stored.

Click the Open the Live Demo button at the top. Nothing to install, and you can start from a built-in sample dataset.

The demo runs on Streamlit Community Cloud and goes to sleep when idle. If you see a sleeping notice, click the wake-up button and wait a few seconds.

You need Python 3.10 or newer.

# 1. Get the code
git clone https://github.com/MahdiKordian/DataForge.git
cd DataForge

# 2. Create an isolated environment (recommended)
python -m venv .venv
source .venv/bin/activate          # Windows: .venv\Scripts\activate

# 3. Install the dependencies
pip install -r requirements.txt

# 4. Launch the app
streamlit run app.py

Streamlit prints a local address, usually http://localhost:8501.

1. Fork this repository.
2. On Streamlit Community Cloud, choose New app and select your fork.
3. Set the main file path to app.py and deploy.

1. Load. On the home page, upload a file or pick a sample. Use the Dataset page for pasted text, a URL, a specific Excel sheet, or to choose a target column.
2. Inspect. Open the Dashboard to read the readiness score and the issue list. Each issue points to the step that fixes it.
3. Prepare. Run Clean, Transform, Encode, Scale and Split in that order. Do the split last so no information leaks between sets.
4. Explore. Use Analysis at any point, and compare Before vs after to see what each step changed.
5. Export. Download the prepared data and the step log, then generate a report.

Tips

• Set a target column on the Dashboard to unlock stratified splitting and class balancing, which apply to classification targets.
• The readiness score rises as you fix issues. It checks for missing values, duplicates, constant columns, outliers (under 1% of values), non-numeric columns and a finished split.
• Loading a new dataset replaces the current one and clears its log.

Three real datasets ship in sample_data/ and are offered on the home page, each with the usual problems so every step has something to do.

The Dataset page also offers three small generated practice sets: customer churn, house prices and student scores.

DataForge is a multi-page Streamlit app. A small shared core keeps one working copy of your data in the session, together with a snapshot after every step and a log of what changed.

• core.py holds the shared state, file readers and helpers. Parsing is cached, and encodings are tried in order until one decodes cleanly.
• Each page in pages/ handles one stage and reads from and writes to the same session state, so you can move freely between pages.
• Every Prepare step runs on the current data, logs row and column changes, and stores a snapshot. Undo restores the previous snapshot, and Reset returns to the data as loaded.
• Reports and exports are generated from the same log and snapshots, so they always match what you actually did.

DataForge/
├── app.py              Entry point: page config, navigation, sidebar
├── core.py             Shared state, file readers, sample loader, helpers
├── pages/
│   ├── landing.py      Home: upload or choose a sample
│   ├── dataset.py      All loading options and target selection
│   ├── dashboard.py    Overview, readiness score, detected issues
│   ├── prepare.py      Clean, Transform, Encode, Scale, Split
│   ├── analysis.py     The nine analysis views
│   ├── export.py       Download data and the step log
│   ├── report.py       HTML and Markdown report builder
│   └── about.py        About the project
├── sample_data/        Ames Housing, Telco Customer Churn, Palmer Penguins
├── docs/               README images
├── requirements.txt    Python dependencies
├── README.md           This file
├── LICENSE             Apache License 2.0
├── NOTICE              Third-party attributions
└── .gitignore

Ideas for future versions. Open an issue if one of these matters to you.

• Save a preparation as a reusable recipe and apply it to new data
• Export the pipeline as a Python script or scikit-learn pipeline
• More encoders, such as target encoding
• Automated tests and continuous integration
• Docker image for one-command deployment

Contributions are welcome, from bug reports to new features.

1. Fork the repository and create a branch: git checkout -b feat/your-idea
2. Make your change and run the app locally to test it.
3. Commit with a clear message. This project follows the Conventional Commits style, for example feat(prepare): add target encoding.
4. Push your branch and open a pull request describing what changed and why.

Found a bug or have a suggestion? Open an issue.

Released under the Apache License 2.0. Third-party attributions are listed in NOTICE.

• Ames Housing by Dean De Cock.
• Palmer Penguins by Dr. Kristen Gorman and the Palmer Station LTER, packaged by Allison Horst and Alison Hill.
• Telco Customer Churn, an IBM sample dataset.
• The open-source libraries listed in the tech stack above.

Please check each dataset's original terms before redistributing it.

Mahdi Kordian, AI Engineer working on machine learning, deep learning, generative AI and LLMs.