Skip to content

Commit 9af0518

Browse files
committed
[feat] #15 Add S3 storage strategy - readme
1 parent a2b6e89 commit 9af0518

File tree

2 files changed

+50
-11
lines changed

2 files changed

+50
-11
lines changed

README.md

Lines changed: 50 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -433,7 +433,7 @@ curl -X POST "http://localhost:8000/llm/generate" -H "Content-Type: application/
433433

434434
The tool can automatically save the results using different storage strategies and storage profiles. Storage profiles are set in the `/storage_profiles` by a yaml configuration files.
435435

436-
Example:
436+
### Local File System
437437

438438
```yaml
439439
strategy: local_filesystem
@@ -443,7 +443,7 @@ settings:
443443
create_subfolders: true
444444
```
445445
446-
for Google drive:
446+
### Google Drive
447447
448448
```yaml
449449
strategy: google_drive
@@ -458,6 +458,54 @@ Where the `service_account_file` is a `json` file with authorization credentials
458458

459459
Note: Service Account is different account that the one you're using for Google workspace (files will not be visible in the UI)
460460

461+
### Amazon S3 - Cloud Object Storage
462+
463+
```yaml
464+
strategy: aws_s3
465+
settings:
466+
bucket_name: ${AWS_S3_BUCKET_NAME}
467+
region: ${AWS_REGION}
468+
access_key: ${AWS_ACCESS_KEY_ID}
469+
secret_access_key: ${AWS_SECRET_ACCESS_KEY}
470+
```
471+
472+
#### Requirements for AWS S3 Access Key
473+
474+
1. **Access Key Ownership**
475+
The access key must belong to an IAM user or role with permissions for S3 operations.
476+
477+
2. **IAM Policy Example**
478+
The IAM policy attached to the user or role must allow the necessary actions. Below is an example of a policy granting access to an S3 bucket:
479+
```json
480+
{
481+
"Version": "2012-10-17",
482+
"Statement": [
483+
{
484+
"Effect": "Allow",
485+
"Action": [
486+
"s3:PutObject",
487+
"s3:GetObject",
488+
"s3:ListBucket",
489+
"s3:DeleteObject"
490+
],
491+
"Resource": [
492+
"arn:aws:s3:::your-bucket-name",
493+
"arn:aws:s3:::your-bucket-name/*"
494+
]
495+
}
496+
]
497+
}
498+
```
499+
500+
And fill the `.env` file with the proper AWS credentials:
501+
502+
```bash
503+
AWS_ACCESS_KEY_ID=your-access-key-id
504+
AWS_SECRET_ACCESS_KEY=your-secret-access-key
505+
AWS_REGION=your-region
506+
AWS_S3_BUCKET_NAME=your-bucket-name
507+
```
508+
461509
## License
462510
This project is licensed under the GNU General Public License. See the [LICENSE](LICENSE.md) file for details.
463511

app/storage_strategies/aws_s3.py

Lines changed: 0 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -4,15 +4,6 @@
44
from botocore.exceptions import NoCredentialsError, ClientError
55
from storage_strategies.storage_strategy import StorageStrategy
66

7-
# Requirements for AWS S3 Access Key:
8-
# 1. The access key must belong to an IAM user or role with permissions for S3 operations.
9-
# 2. The IAM policy attached must explicitly allow actions such as:
10-
# - s3:PutObject (for uploading files)
11-
# - s3:GetObject (for downloading files)
12-
# - s3:ListBucket (for listing files in the bucket)
13-
# - s3:DeleteObject (for deleting files)
14-
# 3. The IAM policy must specify the appropriate resource ARN for the bucket and objects, e.g.:
15-
# - "arn:aws:s3:::your-bucket-name/*" (for all objects in the bucket)
167
class AWSS3StorageStrategy(StorageStrategy):
178
def __init__(self, context):
189
super().__init__(context)

0 commit comments

Comments
 (0)