Streamed JSON Lines

Instead of streaming a single JSON (which it seems does not realy stream with the StreamedJsonResponse class from Symfony/Laravel), they proposed Streamed JSON Lines (https://jsonlines.org/).

Example:

return new \MacropaySolutions\LaravelCrudWizard\ResponsesStreamedJsonResponse(
    Operation::query()->lazyByIdDesc(1000, 'id'),
    encodingOptions: JSON_UNESCAPED_UNICODE | JSON_UNESCAPED_SLASHES
 );

The above will stream each row from DB as a single JSON per line:

{"id":17009,"value":"92.00","created_at":"2024-01-17 09:17:11","updated_at":null,"primary_key_identifier":"17009"} {"id":17008,"value":"87.00","created_at":"2024-01-17 09:17:11","updated_at":null,"primary_key_identifier":"17008"} ... In browser, the following Javascript code could be used to parse the response and display each row as it arrives, thus not needing to wait for the whole JSON stream to finish:

<script> function requestStream(e,form) { e.preventDefault(); const outputElement = document.getElementById("streamed_operations_response"); outputElement.innerHTML = ''; fetch(form.action, {method:'post', headers: { 'Accept': 'application/json' }, body: new URLSearchParams(new FormData(form))}) .then(response => { const reader = response.body.getReader(); const decoder = new TextDecoder(); let leftOver = ''; return new ReadableStream({ start(controller) { function push() { reader.read().then(({ done, value }) => { if (done) { controller.close(); return; } const chunk = decoder.decode(value, { stream: true }); chunk.split(/\n/).forEach(function (element) { let row; try { row = JSON.parse(element); } catch (e) { console.log('e'); if (leftOver === '') { leftOver = element; return; } try { row = JSON.parse(leftOver + element); leftOver = ''; } catch (ex) { console.log('ex'); leftOver += element; console.log('This leftOver should not happen: ' + leftOver); return; } } let child = document.createElement('p'); child.innerHTML = JSON.stringify(row); outputElement.appendChild(child); }); controller.enqueue(value); push(); }); } push(); } }); }) .then(stream => new Response(stream)) .then(response => response.text()) .then(data => { console.log("Streaming complete"); }) .catch(error => { console.error("Streaming error:", error); }); } </script>

A demo can be found here https://laravel-crud-wizard.com/laravel-9/laravel-lumen-crud-wizard#operations. Just select Streamed Json in the Pagination drop-down and submit.

See also StreamedJsonResponse class.