I need to use data in a csv file and transform it into a SQL query that manually insert data row by row for a local data seed.

Even without coding copilot, I still need to rely on AI in a browser for a syntax. So I prompted.

Given a csv file, how can I sed/grep specific columns through terminal?

I used the known terms sed and grep, hoping it understands my context better.

But, to my surprise, it recommended awk, my new learning today.

awk -F, '{print \(1, \)2}' file.csv

This basic yet powerful command allows to to print the first 2 columns of the CSV file.

In my case, my first column is a productId which is string, and my second column price which is numeric. So I can use printf so that it returns my preferred format.

awk -F, {printf("('\''%s'\'', %s),\n", \(1, \)2)} file.csv | tr -d '""'

I also use tr here to remove double quote that I don't need.

Here is what I got

('885100', 99.00),
('885200', 109.00),
...

And yes, I can pretty much copy and paste this into my SQL script to insert product and price, row by row.

If I had tokens left, I would have prompted given this file.csv, extract barcode and price to create a data seed migration. It has been working fine for a while, until I came to realize that my tokens are limited and that I need to rely less on AI.

If you are not running out of tokens (yet), congratulation. But if one day you do, remember awk.

Today, I want to write a SQL script to insert a few hundred barcodes as a data seed for local test.

First I need to some how, extract a list of barcodes from another SQL file that also do insertion, but to another table with a different set of columns

-- old_script.sql
INSERT INTO table_a (...) VALUES ('885100', '000100', ...)

With AI, I would simply added old_script.sql to an AI copilot and then prompt given this file, create another file that insert the similar barcode to table_b

Unfortunately, my company's internal AI credits ran out. And there is a policy against using external / personal AI to read company's code base, as it risks data leakage.

The data scientist version of me might try a Python script to extract the list of barcode, probably with regex, but as I have been observing AI doing their (my) jobs, it often uses faster alternatives - a linux command like sed or grep

So through some examples on the internet, and ChatGPT (without letting it read my code), I came up with

sed -n "s/.*('\([^']*\)', '0.*/\1/p" file.sql

This detects a string between (' and ', '0) and magically list them out line by line

885100
885200
...

Then I modify the command a bit by adding | sed "s/.*/'&',/, so the output becomes

'885100',
'885200',
...

Now I can simply copy these barcodes from Terminal and then continue my quest writing SQL script which, fortunately, I'm familiar with, and I don't need further assistance from AI.