Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions ..bfg-report/2025-06-16/14-24-08/cache-stats.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
(apply,CacheStats{hitCount=1800, missCount=746, loadSuccessCount=738, loadExceptionCount=0, totalLoadTime=1354724079, evictionCount=0})
(tree,CacheStats{hitCount=1557, missCount=924, loadSuccessCount=880, loadExceptionCount=0, totalLoadTime=3455619793, evictionCount=0})
(commit,CacheStats{hitCount=586, missCount=498, loadSuccessCount=498, loadExceptionCount=0, totalLoadTime=334389419, evictionCount=0})
(tag,CacheStats{hitCount=0, missCount=0, loadSuccessCount=0, loadExceptionCount=0, totalLoadTime=0, evictionCount=0})
1 change: 1 addition & 0 deletions ..bfg-report/2025-06-16/14-24-08/changed-files.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
7224a5d82990e7aefe4217a078eb982f6c7edd49 3b074a4d6b12df485cafbc013ca4371e7c859ceb .env
504 changes: 504 additions & 0 deletions ..bfg-report/2025-06-16/14-24-08/object-id-map.old-new.txt

Large diffs are not rendered by default.

111 changes: 111 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
name: Test Python SDK

on:
push:
branches: [ main, master ]
pull_request:
branches: [ main, master ]

jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.8", "3.9", "3.10", "3.11", "3.12"]

steps:
- uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}

- name: Cache pip dependencies
uses: actions/cache@v3
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-${{ matrix.python-version }}-${{ hashFiles('**/pyproject.toml') }}
restore-keys: |
${{ runner.os }}-pip-${{ matrix.python-version }}-

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install pytest pytest-asyncio responses
cd scrapegraph-py
pip install -e .

- name: Run tests with coverage
run: |
cd scrapegraph-py
python -m pytest tests/ -v --cov=scrapegraph_py --cov-report=xml --cov-report=term-missing

- name: Run real API tests (if API key available)
run: |
cd scrapegraph-py
if [ -n "$SGAI_API_KEY" ]; then
python -m pytest tests/test_real_apis.py -v --tb=short
else
echo "SGAI_API_KEY not set, skipping real API tests"
fi
env:
SGAI_API_KEY: ${{ secrets.SGAI_API_KEY }}
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lurenss / @VinciGit00 can you please set env on github?


- name: Upload coverage to Codecov
uses: codecov/codecov-action@v3
with:
file: ./scrapegraph-py/coverage.xml
flags: unittests
name: codecov-umbrella
fail_ci_if_error: false

lint:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: "3.11"

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install flake8 black isort mypy
cd scrapegraph-py
pip install -e .

- name: Run linting
run: |
cd scrapegraph-py
flake8 scrapegraph_py/ tests/ --max-line-length=88 --extend-ignore=E203,W503
black --check scrapegraph_py/ tests/
isort --check-only scrapegraph_py/ tests/
mypy scrapegraph_py/ --ignore-missing-imports

security:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: "3.11"

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install bandit safety
cd scrapegraph-py
pip install -e .

- name: Run security checks
run: |
cd scrapegraph-py
bandit -r scrapegraph_py/ -f json -o bandit-report.json || true
safety check --json --output safety-report.json || true
1 change: 1 addition & 0 deletions api-keys-to-remove.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
sgai-3672f339-d597-45f3-a6f3-d7041fec607b
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: API Key Exposed in Version Control

A ScrapeGraph AI API key (sgai-3672f339-d597-45f3-a6f3-d7041fec607b) was accidentally committed to api-keys-to-remove.txt. Although this file was intended for tracking keys to remove, it contains the literal key, posing a serious security vulnerability by exposing sensitive credentials that should never be in version control.

Fix in Cursor Fix in Web

Binary file added bfg-1.14.0.jar
Binary file not shown.
Loading
Loading