Skip to content

JVM video optimization #140

@shichonghuotian

Description

@shichonghuotian

I tested the playback of 4k videos using the desktop version (JVM) and found that it can only maintain between 10-20 frames per second. After debugging, I found that the conversion of BufferedImage took a lot of time. I have come up with a better solution, After optimization, the frame rate can reach 60 frames per second,you can test and replace it. Here is the code I used. (I'm sorry, I don't have time to submit a PR)

fun updateFrame() {
        val ptr = playerPtr ?: return
        
        try {
            val startTime = System.nanoTime()
            
            val framePtr = MacOSVideoPlayerLib.getLatestFrame(ptr) ?: return
            val jnaTime = (System.nanoTime() - startTime) / 1_000_000.0
            
            val width = MacOSVideoPlayerLib.getFrameWidth(ptr)
            val height = MacOSVideoPlayerLib.getFrameHeight(ptr)
            
            if (width <= 0 || height <= 0) return
            
            val copyStart = System.nanoTime()
            
            // Create or reuse Skia Bitmap
            if (skiaBitmap == null || skiaBitmap!!.width != width || skiaBitmap!!.height != height) {
                val imageInfo = ImageInfo(
                    width = width,
                    height = height,
                    colorType = ColorType.BGRA_8888,
                    alphaType = ColorAlphaType.PREMUL
                )
                skiaBitmap = Bitmap()
                skiaBitmap!!.allocPixels(imageInfo)
                println("[MacOSVideoPlayerController] Created Skia bitmap: ${width}x${height}")
            }
            
            // Copy pixels directly to Skia Bitmap's buffer
            val pixelsAddr = skiaBitmap!!.peekPixels()?.addr ?: return
            val pixelCount = width * height
            val size = pixelCount * 4L
            
            // Use direct ByteBuffers for native-to-native copy
            // This avoids copying to a Java ByteArray
            val srcBuf = framePtr.getByteBuffer(0, size)
            val destPtr = Pointer(pixelsAddr)
            val destBuf = destPtr.getByteBuffer(0, size)
            
            destBuf.put(srcBuf)
            
            val copyTime = (System.nanoTime() - copyStart) / 1_000_000.0
            
            // Convert to ImageBitmap
            _currentFrame.value = skiaBitmap!!.asComposeImageBitmap()
            
            // Calculate FPS
            frameCount++
            val currentTime = System.currentTimeMillis()
            val elapsed = currentTime - lastFpsTime
            if (elapsed >= 1000) {
                _fps.value = (frameCount * 1000.0) / elapsed
                val totalTime = (System.nanoTime() - startTime) / 1_000_000.0
                println("[MacOSVideoPlayerController] FPS: ${String.format("%.1f", _fps.value)} | JNA: ${String.format("%.1f", jnaTime)}ms | Copy: ${String.format("%.1f", copyTime)}ms | Total: ${String.format("%.1f", totalTime)}ms")
                frameCount = 0
                lastFpsTime = currentTime
            }
            
        } catch (e: Exception) {
            println("[MacOSVideoPlayerController] Error updating frame: ${e.message}")
            e.printStackTrace()
        }
    }

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions