Working with CSV data
Create a CSV table and filter rows with Linux tools (grep, awk, sort). The validator reads the result file.
Create and filter a table
Analysing CSV data
CSV ("Comma-Separated Values") is the simplest table format: plain text, columns separated by commas, one line per record. Excel, databases and APIs exchange data this way. You analyse CSV with the Linux tools grep, wc, cut and sort.
Key terms & tools
- Header: the first line with the column names.
grep: filters lines containing a pattern.wc -l: counts lines.cut -d, -f1: cuts out the 1st column (delimiter,).sort: sorts lines alphabetically.
Work in the folder
/home/student/csv/(mkdir -p /home/student/csv).
Your goal
You create a CSV table and analyse it: filter, count, extract columns, sort.
Exercises
1. Create a CSV
Concept: CSV structure. First line = column names, then one line per record. Create a table
schueler.csvwith the headername,klasse,alterand at least four data lines.mkdir -p /home/student/csv printf "name,klasse,alter\nMax,7a,13\nLisa,7b,12\nTom,7a,13\nAnna,8a,14\n" > /home/student/csv/schueler.csvCheck:
schueler.csvstarts withname,klasse,alterand has at least 4 data lines with 3 columns each.2. Filter for class 7a
Concept: filtering lines with
grep. Extract all students of class7aand write them toklasse-7a.csv.grep ",7a," /home/student/csv/schueler.csv > /home/student/csv/klasse-7a.csvThe pattern
,7a,matches the class column exactly (between two commas).Check:
klasse-7a.csvcontains lines with,7a,.3. Total number of students
Concept: counting lines. Count the data lines (excluding the header) and write only the number to
anzahl.txt.tail -n +2skips the first line.tail -n +2 /home/student/csv/schueler.csv | wc -l > /home/student/csv/anzahl.txtCheck:
anzahl.txtcontains exactly one number.4. Extract names only
Concept: extracting a column with
cut.cut -d, -f1cuts out the 1st column (comma delimiter). Get only the names (without header) intonamen.txt.tail -n +2 /home/student/csv/schueler.csv | cut -d, -f1 > /home/student/csv/namen.txtCheck:
namen.txtcontains at least four lines, each with one name.5. Sort students alphabetically
Concept: sorting while keeping the header. Sort the students alphabetically by name, but keep the header on top. Trick: output the header separately, sort the rest.
( head -1 /home/student/csv/schueler.csv; tail -n +2 /home/student/csv/schueler.csv | sort ) > /home/student/csv/sortiert.csvCheck:
sortiert.csvstarts with the headername,klasse,alter, followed by the sorted lines.
Related courses
Now practice it yourself
Reading is good – doing is better. Start this course on a real Linux VM, right in your browser. A free account is all it takes.
Start for freeLab content under CC BY 4.0 – free to use with attribution (© TechLogia).
