Skip to content

Conversation

@cathteng
Copy link
Member

@cathteng cathteng commented Nov 6, 2025

Since we now have DetectorGroup rows being written, we could refactor the WorkflowGroupHistory API to use DetectorGroup, rather than use a column on WorkflowFireHistory (which shouldn't need to know which detector triggered the workflow to be fired).

Don't use DetectorWorkflow to check for active detector - workflow connections because a detector might have been disconnected from the workflow since the workflow was triggered for that issue.

@cathteng cathteng requested a review from a team as a code owner November 6, 2025 21:38
@github-actions github-actions bot added the Scope: Backend Automatically applied to PRs that change backend components label Nov 6, 2025
detector_subquery = DetectorGroup.objects.filter(
group=OuterRef("group"),
detector_id__in=DetectorWorkflow.objects.filter(workflow=workflow).values("detector_id"),
).values("detector_id")[:1]

This comment was marked as outdated.

@cathteng
Copy link
Member Author

cathteng commented Nov 7, 2025

Having second thoughts on whether we should do this because the detector in the notification is so tightly coupled with the metric detector code (charts, open periods, etc)

@cathteng cathteng marked this pull request as draft November 7, 2025 19:03
@cathteng cathteng marked this pull request as ready for review November 7, 2025 23:06
@cathteng
Copy link
Member Author

cathteng commented Nov 7, 2025

OK we back. We will decouple detector from action.trigger here #103000

Comment on lines 131 to 136
group=OuterRef("group"),
detector_id__in=DetectorWorkflow.objects.filter(workflow=workflow).values("detector_id"),
).values("detector_id")[:1]

qs = (
filtered_history.select_related("group", "detector")
filtered_history.select_related("group")
.values("group")
.annotate(count=Count("group"))
.annotate(event_id=Subquery(group_max_dates.values("event_id")))
.annotate(last_triggered=Max("date_added"))
.annotate(detector_id=Subquery(group_max_dates.values("detector_id")))
.annotate(detector_id=Subquery(detector_subquery))
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: The DetectorGroup subquery fails for old groups lacking DetectorGroup entries, causing incorrect detector_id reporting.
Severity: CRITICAL | Confidence: 0.98

🔍 Detailed Analysis

The detector_subquery in workflow_group_history_serializer.py assumes that a DetectorGroup entry exists for every WorkflowFireHistory record with a detector_id. This assumption is false for groups created before the introduction of DetectorGroup entries. When a workflow fires for such an 'old' group, a WorkflowFireHistory record is created with a detector_id, but no corresponding DetectorGroup entry exists. Consequently, the subquery returns None, leading the API to incorrectly report detector=None for these groups, despite a detector having fired the workflow.

💡 Suggested Fix

Modify the detector_subquery in workflow_group_history_serializer.py to correctly retrieve the detector_id for groups that lack a DetectorGroup entry, potentially by falling back to WorkflowFireHistory.detector_id or ensuring DetectorGroup entries are backfilled.

🤖 Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent.
Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not
valid.

Location:
src/sentry/workflow_engine/endpoints/serializers/workflow_group_history_serializer.py#L130-L141

Potential issue: The `detector_subquery` in `workflow_group_history_serializer.py`
assumes that a `DetectorGroup` entry exists for every `WorkflowFireHistory` record with
a `detector_id`. This assumption is false for groups created before the introduction of
`DetectorGroup` entries. When a workflow fires for such an 'old' group, a
`WorkflowFireHistory` record is created with a `detector_id`, but no corresponding
`DetectorGroup` entry exists. Consequently, the subquery returns `None`, leading the API
to incorrectly report `detector=None` for these groups, despite a detector having fired
the workflow.

Did we get this right? 👍 / 👎 to inform future reviews.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We expect groups that don't have DetectorGroup entries to be attributed to the issue stream detector. You also can't click into the issue stream detector in the UI

Comment on lines +125 to +128
detector_subquery = DetectorGroup.objects.filter(
group=OuterRef("group"),
).values(
"detector_id"
)[:1]

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

High severity vulnerability may affect your project—review required:
Line 125 lists a dependency (django) with a known High severity vulnerability.

ℹ️ Why this matters

Affected versions of Django are vulnerable to Improper Neutralization of Special Elements used in an SQL Command ('SQL Injection'). SQL injection in Django's ORM column aliases: when using QuerySet.annotate(), QuerySet.alias(), QuerySet.aggregate(), or QuerySet.extra() with dictionary expansion (**kwargs), the dictionary keys are used unescaped as SQL column aliases. On MySQL and MariaDB backends, an attacker who can influence those keys (for example, by passing a crafted dict of annotations) can inject arbitrary SQL into the generated query.

References: GHSA, CVE

To resolve this comment:
Check if you are using Django with MySQL or MariaDB.

  • If you're affected, upgrade this dependency to at least version 5.2.7 at uv.lock.
  • If you're not affected, comment /fp we don't use this [condition]
💬 Ignore this finding

To ignore this, reply with:

  • /fp <comment> for false positive
  • /ar <comment> for acceptable risk
  • /other <comment> for all other reasons

You can view more details on this finding in the Semgrep AppSec Platform here.

@cathteng cathteng force-pushed the cathy/aci/decouple-detector-workflowfirehistory branch from de1b9cc to 73205e2 Compare November 20, 2025 17:12
@codecov
Copy link

codecov bot commented Nov 20, 2025

❌ 3 Tests Failed:

Tests completed Failed Passed Skipped
29907 3 29904 244
View the top 3 failed test(s) by shortest run time
tests.sentry.workflow_engine.endpoints.test_organization_workflow_group_history.WorkflowGroupHistoryEndpointTest::test_invalid_dates_error
Stack Traces | 1.87s run time
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/backends/utils.py#x1B[0m:105: in _execute
    return self.cursor.execute(sql, params)
#x1B[1m#x1B[.../db/postgres/decorators.py#x1B[0m:16: in inner
    return func(self, *args, **kwargs)
#x1B[1m#x1B[.../db/postgres/base.py#x1B[0m:95: in execute
    return self.cursor.execute(sql, params)
#x1B[1m#x1B[31mE   psycopg2.errors.SyntaxError: multiple assignments to same column "project_id"#x1B[0m

#x1B[33mThe above exception was the direct cause of the following exception:#x1B[0m
#x1B[1m#x1B[.../workflow_engine/endpoints/test_organization_workflow_group_history.py#x1B[0m:32: in setUp
    self.detector = self.create_detector(
#x1B[1m#x1B[.../sentry/testutils/fixtures.py#x1B[0m:705: in create_detector
    return Factories.create_detector(project=project, type=type, *args, **kwargs)
#x1B[1m#x1B[.../hostedtoolcache/Python/3.13.1.../x64/lib/python3.13/contextlib.py#x1B[0m:85: in inner
    return func(*args, **kwds)
#x1B[1m#x1B[.../sentry/testutils/factories.py#x1B[0m:2294: in create_detector
    detector.update(config=config, name=name, **kwargs)
#x1B[1m#x1B[.../sentry/silo/base.py#x1B[0m:159: in override
    return original_method(*args, **kwargs)
#x1B[1m#x1B[.../db/models/query.py#x1B[0m:92: in update
    instance.__class__.objects.using(using)
#x1B[1m#x1B[.../sentry/silo/base.py#x1B[0m:159: in override
    return original_method(*args, **kwargs)
#x1B[1m#x1B[.../models/manager/base_query_set.py#x1B[0m:89: in update
    return super().update(**kwargs)
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/models/query.py#x1B[0m:1263: in update
    rows = query.get_compiler(self.db).execute_sql(ROW_COUNT)
#x1B[1m#x1B[31m.venv/lib/python3.13.../models/sql/compiler.py#x1B[0m:2060: in execute_sql
    row_count = super().execute_sql(result_type)
#x1B[1m#x1B[31m.venv/lib/python3.13.../models/sql/compiler.py#x1B[0m:1623: in execute_sql
    cursor.execute(sql, params)
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/backends/utils.py#x1B[0m:122: in execute
    return super().execute(sql, params)
#x1B[1m#x1B[31m.venv/lib/python3.13.../site-packages/sentry_sdk/utils.py#x1B[0m:1849: in runner
    return original_function(*args, **kwargs)
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/backends/utils.py#x1B[0m:79: in execute
    return self._execute_with_wrappers(
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/backends/utils.py#x1B[0m:92: in _execute_with_wrappers
    return executor(sql, params, many, context)
#x1B[1m#x1B[.../db/postgres/base.py#x1B[0m:70: in _execute__include_sql_in_error
    return execute(sql, params, many, context)
#x1B[1m#x1B[.../db/postgres/base.py#x1B[0m:58: in _execute__clean_params
    return execute(sql, clean_bad_params(params), many, context)
#x1B[1m#x1B[.../sentry/testutils/hybrid_cloud.py#x1B[0m:133: in __call__
    return execute(*params)
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/backends/utils.py#x1B[0m:100: in _execute
    with self.db.wrap_database_errors:
#x1B[1m#x1B[31m.venv/lib/python3.13.../django/db/utils.py#x1B[0m:91: in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/backends/utils.py#x1B[0m:105: in _execute
    return self.cursor.execute(sql, params)
#x1B[1m#x1B[.../db/postgres/decorators.py#x1B[0m:16: in inner
    return func(self, *args, **kwargs)
#x1B[1m#x1B[.../db/postgres/base.py#x1B[0m:95: in execute
    return self.cursor.execute(sql, params)
#x1B[1m#x1B[31mE   django.db.utils.ProgrammingError: multiple assignments to same column "project_id"#x1B[0m
#x1B[1m#x1B[31mE   #x1B[0m
#x1B[1m#x1B[31mE   SQL: UPDATE "workflow_engine_detector" SET "config" = %s, "name" = %s, "project_id" = %s, "type" = %s, "project_id" = %s, "date_updated" = %s WHERE (NOT ("workflow_engine_detector"."status" IN (%s, %s)) AND "workflow_engine_detector"."id" = %s)#x1B[0m
tests.sentry.workflow_engine.endpoints.test_organization_workflow_group_history.WorkflowGroupHistoryEndpointTest::test_pagination
Stack Traces | 2.27s run time
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/backends/utils.py#x1B[0m:105: in _execute
    return self.cursor.execute(sql, params)
#x1B[1m#x1B[.../db/postgres/decorators.py#x1B[0m:16: in inner
    return func(self, *args, **kwargs)
#x1B[1m#x1B[.../db/postgres/base.py#x1B[0m:95: in execute
    return self.cursor.execute(sql, params)
#x1B[1m#x1B[31mE   psycopg2.errors.SyntaxError: multiple assignments to same column "project_id"#x1B[0m

#x1B[33mThe above exception was the direct cause of the following exception:#x1B[0m
#x1B[1m#x1B[.../workflow_engine/endpoints/test_organization_workflow_group_history.py#x1B[0m:32: in setUp
    self.detector = self.create_detector(
#x1B[1m#x1B[.../sentry/testutils/fixtures.py#x1B[0m:705: in create_detector
    return Factories.create_detector(project=project, type=type, *args, **kwargs)
#x1B[1m#x1B[.../hostedtoolcache/Python/3.13.1.../x64/lib/python3.13/contextlib.py#x1B[0m:85: in inner
    return func(*args, **kwds)
#x1B[1m#x1B[.../sentry/testutils/factories.py#x1B[0m:2294: in create_detector
    detector.update(config=config, name=name, **kwargs)
#x1B[1m#x1B[.../sentry/silo/base.py#x1B[0m:159: in override
    return original_method(*args, **kwargs)
#x1B[1m#x1B[.../db/models/query.py#x1B[0m:92: in update
    instance.__class__.objects.using(using)
#x1B[1m#x1B[.../sentry/silo/base.py#x1B[0m:159: in override
    return original_method(*args, **kwargs)
#x1B[1m#x1B[.../models/manager/base_query_set.py#x1B[0m:89: in update
    return super().update(**kwargs)
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/models/query.py#x1B[0m:1263: in update
    rows = query.get_compiler(self.db).execute_sql(ROW_COUNT)
#x1B[1m#x1B[31m.venv/lib/python3.13.../models/sql/compiler.py#x1B[0m:2060: in execute_sql
    row_count = super().execute_sql(result_type)
#x1B[1m#x1B[31m.venv/lib/python3.13.../models/sql/compiler.py#x1B[0m:1623: in execute_sql
    cursor.execute(sql, params)
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/backends/utils.py#x1B[0m:122: in execute
    return super().execute(sql, params)
#x1B[1m#x1B[31m.venv/lib/python3.13.../site-packages/sentry_sdk/utils.py#x1B[0m:1849: in runner
    return original_function(*args, **kwargs)
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/backends/utils.py#x1B[0m:79: in execute
    return self._execute_with_wrappers(
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/backends/utils.py#x1B[0m:92: in _execute_with_wrappers
    return executor(sql, params, many, context)
#x1B[1m#x1B[.../db/postgres/base.py#x1B[0m:70: in _execute__include_sql_in_error
    return execute(sql, params, many, context)
#x1B[1m#x1B[.../db/postgres/base.py#x1B[0m:58: in _execute__clean_params
    return execute(sql, clean_bad_params(params), many, context)
#x1B[1m#x1B[.../sentry/testutils/hybrid_cloud.py#x1B[0m:133: in __call__
    return execute(*params)
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/backends/utils.py#x1B[0m:100: in _execute
    with self.db.wrap_database_errors:
#x1B[1m#x1B[31m.venv/lib/python3.13.../django/db/utils.py#x1B[0m:91: in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/backends/utils.py#x1B[0m:105: in _execute
    return self.cursor.execute(sql, params)
#x1B[1m#x1B[.../db/postgres/decorators.py#x1B[0m:16: in inner
    return func(self, *args, **kwargs)
#x1B[1m#x1B[.../db/postgres/base.py#x1B[0m:95: in execute
    return self.cursor.execute(sql, params)
#x1B[1m#x1B[31mE   django.db.utils.ProgrammingError: multiple assignments to same column "project_id"#x1B[0m
#x1B[1m#x1B[31mE   #x1B[0m
#x1B[1m#x1B[31mE   SQL: UPDATE "workflow_engine_detector" SET "config" = %s, "name" = %s, "project_id" = %s, "type" = %s, "project_id" = %s, "date_updated" = %s WHERE (NOT ("workflow_engine_detector"."status" IN (%s, %s)) AND "workflow_engine_detector"."id" = %s)#x1B[0m
tests.sentry.workflow_engine.endpoints.test_organization_workflow_group_history.WorkflowGroupHistoryEndpointTest::test_simple
Stack Traces | 2.29s run time
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/backends/utils.py#x1B[0m:105: in _execute
    return self.cursor.execute(sql, params)
#x1B[1m#x1B[.../db/postgres/decorators.py#x1B[0m:16: in inner
    return func(self, *args, **kwargs)
#x1B[1m#x1B[.../db/postgres/base.py#x1B[0m:95: in execute
    return self.cursor.execute(sql, params)
#x1B[1m#x1B[31mE   psycopg2.errors.SyntaxError: multiple assignments to same column "project_id"#x1B[0m

#x1B[33mThe above exception was the direct cause of the following exception:#x1B[0m
#x1B[1m#x1B[.../workflow_engine/endpoints/test_organization_workflow_group_history.py#x1B[0m:32: in setUp
    self.detector = self.create_detector(
#x1B[1m#x1B[.../sentry/testutils/fixtures.py#x1B[0m:705: in create_detector
    return Factories.create_detector(project=project, type=type, *args, **kwargs)
#x1B[1m#x1B[.../hostedtoolcache/Python/3.13.1.../x64/lib/python3.13/contextlib.py#x1B[0m:85: in inner
    return func(*args, **kwds)
#x1B[1m#x1B[.../sentry/testutils/factories.py#x1B[0m:2294: in create_detector
    detector.update(config=config, name=name, **kwargs)
#x1B[1m#x1B[.../sentry/silo/base.py#x1B[0m:159: in override
    return original_method(*args, **kwargs)
#x1B[1m#x1B[.../db/models/query.py#x1B[0m:92: in update
    instance.__class__.objects.using(using)
#x1B[1m#x1B[.../sentry/silo/base.py#x1B[0m:159: in override
    return original_method(*args, **kwargs)
#x1B[1m#x1B[.../models/manager/base_query_set.py#x1B[0m:89: in update
    return super().update(**kwargs)
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/models/query.py#x1B[0m:1263: in update
    rows = query.get_compiler(self.db).execute_sql(ROW_COUNT)
#x1B[1m#x1B[31m.venv/lib/python3.13.../models/sql/compiler.py#x1B[0m:2060: in execute_sql
    row_count = super().execute_sql(result_type)
#x1B[1m#x1B[31m.venv/lib/python3.13.../models/sql/compiler.py#x1B[0m:1623: in execute_sql
    cursor.execute(sql, params)
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/backends/utils.py#x1B[0m:122: in execute
    return super().execute(sql, params)
#x1B[1m#x1B[31m.venv/lib/python3.13.../site-packages/sentry_sdk/utils.py#x1B[0m:1849: in runner
    return original_function(*args, **kwargs)
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/backends/utils.py#x1B[0m:79: in execute
    return self._execute_with_wrappers(
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/backends/utils.py#x1B[0m:92: in _execute_with_wrappers
    return executor(sql, params, many, context)
#x1B[1m#x1B[.../db/postgres/base.py#x1B[0m:70: in _execute__include_sql_in_error
    return execute(sql, params, many, context)
#x1B[1m#x1B[.../db/postgres/base.py#x1B[0m:58: in _execute__clean_params
    return execute(sql, clean_bad_params(params), many, context)
#x1B[1m#x1B[.../sentry/testutils/hybrid_cloud.py#x1B[0m:133: in __call__
    return execute(*params)
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/backends/utils.py#x1B[0m:100: in _execute
    with self.db.wrap_database_errors:
#x1B[1m#x1B[31m.venv/lib/python3.13.../django/db/utils.py#x1B[0m:91: in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/backends/utils.py#x1B[0m:105: in _execute
    return self.cursor.execute(sql, params)
#x1B[1m#x1B[.../db/postgres/decorators.py#x1B[0m:16: in inner
    return func(self, *args, **kwargs)
#x1B[1m#x1B[.../db/postgres/base.py#x1B[0m:95: in execute
    return self.cursor.execute(sql, params)
#x1B[1m#x1B[31mE   django.db.utils.ProgrammingError: multiple assignments to same column "project_id"#x1B[0m
#x1B[1m#x1B[31mE   #x1B[0m
#x1B[1m#x1B[31mE   SQL: UPDATE "workflow_engine_detector" SET "config" = %s, "name" = %s, "project_id" = %s, "type" = %s, "project_id" = %s, "date_updated" = %s WHERE (NOT ("workflow_engine_detector"."status" IN (%s, %s)) AND "workflow_engine_detector"."id" = %s)#x1B[0m

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

group=OuterRef("group"),
).values(
"detector_id"
)[:1]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens with this subquery when there isn't a DetectorGroup? Groups do exist visibly before DetectorGroup for a short period, but also our backfill isn't quite finished.
This is my only concern at this point.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will return None and the frontend will assume it's associated with the issue stream detector (maybe not implemented yet but it will)

@cathteng cathteng merged commit ac5f25e into master Nov 21, 2025
65 of 67 checks passed
@cathteng cathteng deleted the cathy/aci/decouple-detector-workflowfirehistory branch November 21, 2025 18:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Scope: Backend Automatically applied to PRs that change backend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants