Skip to content

Conversation

@aeroith
Copy link

@aeroith aeroith commented Nov 21, 2025

Implements a new API endpoint that retrieves a specific keyframe by its UUID and returns the raw frame bytes with metadata in response headers:

  • Content-Type based on codec (video/h264, video/hevc, image/jpeg, etc.)
  • X-Frame-UUID, X-Frame-Timestamp-NS, X-Frame-Codec
  • X-Frame-Width, X-Frame-Height, X-Frame-Keyframe

Implements a new API endpoint that retrieves a specific keyframe by its UUID
and returns the raw frame bytes with metadata in response headers:
- Content-Type based on codec (video/h264, video/hevc, image/jpeg, etc.)
- X-Frame-UUID, X-Frame-Timestamp-NS, X-Frame-Codec
- X-Frame-Width, X-Frame-Height, X-Frame-Keyframe

Includes FrameData struct, Store trait method, RocksDB implementation,
service layer integration, and comprehensive unit tests.
@bwsw
Copy link
Contributor

bwsw commented Nov 24, 2025

@aeroith thank you for the PR. I would like to comment on specific points that concern me:

Data return and structure. You expect that the first element contains frame content, which is not true. The content can be embedded in the frame itself, kept in the first data item, or stored in a 3rd-party system like KeyDb, where only a reference is stored in the frame or data. The implementation depends on adapters and their negotiation.

Thus, in general, you cannot determine where the data is without frame metadata. Therefore, you need to return [frame_metadata, all, data, items]. Probably, the best way to do that is to use multipart/mixed and return everything. For the metadata, you can implement an API to request either json (videoframe.json()) or native format, which means a protobuf-serialized Message.

The current implementation with headers and the first data item is not universally applicable and cannot be reliably used in systems that use other encoding schemes. Please take a look at our Kafka/Redis adapter for an example: frame contents are kept in Redis/KeyDB, while metadata travels over Kafka, containing only a reference to the storage. This is beneficial for systems that rely on data deduplication.

Keyframe limitation. You implemented it to request only keyframes; however, it would be better to support any frame. Keyframes are a logical concept, containing just a starting moment when you can decode a frame sequence. E.g., for JPEGs, you do not need to set KF on every JPEG, but also on some of them, like 1 per second or dynamically based on particular conditions (e.g., the scene started to contain something useful). Thus, KF (in the context of Replay) only defines the size of the seek index and may not reflect the 'autonomous decoding' property of a particular element.

@aeroith
Copy link
Author

aeroith commented Nov 24, 2025

@bwsw Thank you so much. It makes much more sense now to me. I'll implement these changes later today, and let you know if I need more input.

@aeroith
Copy link
Author

aeroith commented Nov 25, 2025

Hi @bwsw, correct me if I'm wrong, I've never used rocksdb in the past but from what I see here rocksdb.rs it seems only keyframes are indexed. Implementing this function to support all frames would be very inefficient from what I see.

@bwsw
Copy link
Contributor

bwsw commented Nov 25, 2025

@aeroith let me check. Maybe I misinformed you.

@bwsw
Copy link
Contributor

bwsw commented Nov 25, 2025

@aeroith yes, I misinformed you. I thought we used UUIDs, not incremented values, but it seems we did not. Please, implement only for keyframes that are available in the kf index.

@aeroith
Copy link
Author

aeroith commented Nov 30, 2025

No worries. I'll send an update with changes in the response side.

 - Return JSON metadata + binary data parts instead
  - Add tests
  - Update API docs
@aeroith
Copy link
Author

aeroith commented Nov 30, 2025

Hi @bwsw,

Updated the implementation based on your feedback:
Response structure
Now returns multipart/mixed with:

  • Part 1: JSON metadata (VideoFrame.to_serde_json_value())
  • Parts 2+: All data items with Content-Disposition: inline; name="data"; index="N"

Removed

  • Old X-Frame-* headers (timestamp, width, height, codec) - metadata is now in the multipart body

Note:
I didn't implement the native (protobuf) format since all other REST endpoints return JSON only. If you'd prefer, I can add format=protobuf query parameter that would serialize the full Message so other clients can use it like kafka / redis adapter.

Also, this is the first time I actually implemented a multipart/mixed response. I read the RFC but let me know if you have concerns.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants