CSV Profile And Sanity Check

1---
2name: csv-profile-and-sanity-check
3description: Reads a CSV or spreadsheet export without changing it and reports what is in it, including row and column counts, each column's type, blanks, distinct values, numeric ranges, duplicates, mixed formats, odd outliers and case or spacing variants, then lists what to ask the data's owner before anyone uses it. Use when someone has a CSV or exported sheet and says "what is in this file", "is this data OK", "sanity check this export", "profile this CSV", or before cleaning, joining or analysing a file.
4---
5 
6# CSV profile and sanity check
7 
8You describe a data file honestly before anyone builds on it. You never edit the file.
9 
10## What you need
111. The **file**: a CSV (or a sheet exported to CSV). If you can read files from the user's folder, use the path. Otherwise ask the user to paste the header and a sample of rows, and say how many rows there are in total.
122. Optional: what the file is meant to contain, and what it will be used for.
13 
14**Stop condition.** If you can read only a header and no data rows, report the header, say a profile is not possible, and stop.
15 
16## Method
171. **If you can run Python 3**, run `scripts/profile_csv.py <file>` (standard library only; it only reads). Use its output as the facts. It reports encoding, delimiter, row and column counts, ragged rows, exact duplicate rows, and for each column its type, blanks, distinct count, numeric range, common values, and flags such as mixed types, dates in more than one format, values with stray spaces, case variants, constant columns, outliers (3 x IQR) and leading zeros.
182. **If you cannot run code**, say so. Work from what the user pasted: the header and a sample. State that the profile is from a sample of N rows and may miss problems elsewhere. Check the same things by eye and do not give counts for the whole file. Never estimate totals you cannot see.
193. **Read the output as an analyst.** For each flag, say what it could mean. For example: dates in two formats may be day and month swapped; a numeric column with a far outlier may be a typo or a unit mix; an ID column with duplicates may be a join problem; leading zeros mean the column should be read as text.
204. **Compare with the purpose**, if the user gave one: which columns matter for it, and which problems would hurt it.
21 
22## Output
23**Snapshot:** rows, columns, encoding, delimiter, duplicates, ragged rows (or "from a sample of N rows" if you could not run the script).
24 
25**Column table:** column | type | blank % | distinct | range or common values | note.
26 
27**Flags, most serious first:** what you saw, why it matters, how to check it (a filter, a count, a question), never a fix applied.
28 
29**Questions for the data's owner:** five to eight, specific to this file.
30 
31**Not checked:** what a profile cannot tell you (whether values are true, whether rows are missing, where the data came from).
32 
33## Rules
34- Read only. Do not write, sort, rename or "clean" the file. Suggest steps; do not perform them.
35- Do not bring in outside facts about what the data describes. If something looks incomplete (for example, a total that seems low), ask the owner.
36- Do not guess the meaning of a column. If a name is unclear, list it under questions.
37- Report numbers from the script or from rows you can see. Do not invent counts.
38- Plain, short sentences. No em dashes.
39