Category : amazon-s3

I’m using AWS CLI tool to download hundreds of thousands of files. I have almost a million of these one-liners generated from SQL query with different file path that I need to go through: aws s3 cp s3://[myS3FilePath]/17802c9-6d3b-4eef-855a-a6ae0039c7ff/ C:[MyLocalFilePath]802c9-6d3b-4eef-855a-a6ae0039c7ff –recursive I’ve been taking ~1000 lines at a time, pasting them to command prompt, and waiting ..

Read more

I am installing the aws-cpp-sdk in Ubuntu 20.04 by following the installation instructions on: https://github.com/aws/aws-sdk-cpp I built/installed the SDK from source as per the instructions which worked. Installation locations: /usr/local/lib /usr/local/include/aws I built/installed the aws-c-common as per the instructions (although I didn’t use a DCMAKE_INSTALL_PREFIX). I attempt to build aws-checksums but can’t get past the ..

Read more

For my C++ programs that handle large amounts of data in S3 I copy the data down to a local hard drive and then process them in C++ using fopen, fread, etc. Is there a fast way to read an S3 file (object) directly into RAM? Where you can stream right into application memory? I’ve ..

Read more

I’m trying to attach a function to upload a file directly to my bucket in s3 on my UWP’s Application, but I’m getting this error: Unable to parse ExceptionName: BadRequest Message: An error occurred when parsing the HTTP request Here’s my S3 upload function code: { Aws::Client::ClientConfiguration config; config.region = Aws::Region::SA_EAST_1; Aws::S3::S3Client s3_client(credentials, config); Aws::S3::Model::PutObjectRequest ..

Read more

Basically I have the same problem described in this question. Except I an using C++ with opensll instead of Python. When I am trying to create a signature from the example. I get the non-hex version of the signature in the final step. 3431315da57da4df28f92895c75364d94b36c745896ad3e580c0a6ae403b1e05 In the docs the hex version looks like this: 3ce5b2f2fffac9262b4da9256f8d086b4aaf42eba5f111c21681a65a127b7c2a From ..

Read more

I try to create an s3 client with the AWS s3 SDK CPP, I used my own server, and I try to give the client the access key id and the secretKey. my code is: #include <aws/core/Aws.h> #include <aws/s3/S3Client.h> #include <aws/core/auth/AWSCredentialsProvider.h> Aws::Client::ClientConfiguration config; config.scheme = Aws::Http::Scheme::HTTP; string server = s3_host + ":" + s3_port; config.endpointOverride ..

Read more

I am quite new to the C++ world, and I am trying to write a file stream to a disk.I am able to do the following: const Aws::String objectKey2("filename.tar"); Aws::S3::S3Client s3_client2; Aws::S3::Model::GetObjectRequest object_request2; object_request2.SetBucket(bucket_name); object_request2.SetKey(objectKey2); Aws::S3::Model::GetObjectOutcome get_object_outcome2 = s3_client2.GetObject(object_request2); auto &retrieved_file2 = get_object_outcome2.GetResult().GetBody(); std::ios_base::sync_with_stdio(false); auto myfile = std::fstream("file.tar", std::ios::out | std::ios::binary); myfile.write((char*) &retrieved_file2, 1024*100); myfile.close(); ..

Read more