I’m using AWS CLI tool to download hundreds of thousands of files. I have almost a million of these one-liners generated from SQL query with different file path that I need to go through:
aws s3 cp s3://[myS3FilePath]/17802c9-6d3b-4eef-855a-a6ae0039c7ff/ C:[MyLocalFilePath]802c9-6d3b-4eef-855a-a6ae0039c7ff --recursive
I’ve been taking ~1000 lines at a time, pasting them to command prompt, and waiting for them to be iterated through. Works great!
It’s quite a time waste doing it in 1000 record batches though. What’s the maximum amount of lines that I could paste to CMD without losing any of my download commands?
Could I paste 1.000.000 lines into the command prompt for example and trust that it will iterate through all of them?
Source: Windows Questions