Trying to achieve a good 30 fps on Pi 3 Model B using the old 5MP camera module 1.3. Recently downloaded OS Bullseye, and using PiCamera2 on Python.
The goal is to achieve:
- Good FPS ~30FPS for a 100~200 frames consequent captures
- Synchronise a LED flash at start of each frame
We have already read through other posts in which I will try to list here:
- Do not write each frame to the disk after each capture, as writing is a slow I/O process. Instead write to memory, then write to disk when done. Inspiration came from here Take images in a short time using the Raspberry Pi camera module
- Use threading if possible, although we haven't tried that yet! Inspiration from here increasing raspberry p fps with python and although written for PiCamera & openCV libraries and not PiCamera2
- It's a good idea to capture a video rather than still images, as it is much faster and can utilise the full capacity of the camera hardware of different video modes with different FPS options based on resolution
These options provide good direction overall, but it is sill unclear (nor were we able to find it documented in the PiCamera2 docs) how to sync each frame with a callback, or a way to process a frame after/before it is complete and ready to start or end.
Using the information given in the previously mentioned docs and links (probably others too that we might not recall right away) we created a code that was able to reach ~14 FPS @1296x972 in still images capture by writing to BytesIO variable in a while loop then saving in a later loop when all frames are captured. However, achieving the intended FPS is still a challenge, as well as syncing the LED flash still seem to have some hit & miss occasions.
To achieve this FPS we had to test a few configurations, resolutions and formats! We noticed, what seems to be strange to us, not to say we are any images format & encoders experts, that capturing using camera.capture_file(data, format='bmp') then saving using byte_image.save("file_"+str(k).zfill(2)+".jpg", "JPEG") somehow pushed the FPS to be more efficient!
Below are the three tests we made so far:
- Capture JPEG, save JPG --> slow fps (~8 fps)
- Capture BMP, save BMP --> much better fps (~14 fps), but very slow write, also larger file size (~3.6MB)
- Capture BMP, save JPG --> faster fps (~14 fps), good file size (~90KB)
Results:
- Capture JPEG, save JPG --> slow fps (~8 fps)
# CAPTURE JPEG, WRITE JPG
# camera.capture_file(data, format='jpeg')
# byte_image.save("file_jpg"+str(k).zfill(2)+".jpg", "JPEG")
# FPS: 8.296
# frame capture time: ~110ms
# write speed: ok`
[0:49:10.294702270] [1722] INFO Camera camera_manager.cpp:297 libcamera v0.0.5+83-bde9b04f
[0:49:10.342869523] [1723] INFO RPI vc4.cpp:437 Registered camera /base/soc/i2c0mux/i2c@1/ov5647@36 to Unicam device /dev/media2 and ISP device /dev/media0
[0:49:10.342984934] [1723] INFO RPI pipeline_base.cpp:1101 Using configuration file '/usr/share/libcamera/pipeline/rpi/vc4/rpi_apps.yaml'
[0:49:10.356715810] [1722] INFO Camera camera.cpp:1033 configuring streams: (0) 1296x972-XBGR8888 (1) 1296x972-SGBRG10_CSI2P
[0:49:10.357403950] [1723] INFO RPI vc4.cpp:565 Sensor: /base/soc/i2c0mux/i2c@1/ov5647@36 - Selected sensor format: 1296x972-SGBRG10_1X10 - Selected unicam format: 1296x972-pGAA
Capturing..
0 time: 0.16273784637451172
1 time: 0.1138772964477539
2 time: 0.12007522583007812
3 time: 0.11526846885681152
4 time: 0.11428046226501465
5 time: 0.11603045463562012
6 time: 0.1139824390411377
7 time: 0.11432242393493652
8 time: 0.11590003967285156
9 time: 0.11477231979370117
8.295 fps, 10 images
Saving..(total 10 images)
Finished
- Capture BMP, save BMP much better fps (~14 fps), but very slow write, also larger file size (~3.6MB)
# CAPTURE BMP, WRITE BMP
# camera.capture_file(data, format='bmp')
# byte_image.save("file_bmp"+str(k).zfill(2)+".bmp", "BMP")
# FPS: 14.327
# frame capture time: ~62ms
# write speed: SLOW
[0:39:10.717184668] [1609] INFO Camera camera_manager.cpp:297 libcamera v0.0.5+83-bde9b04f
[0:39:10.765629983] [1610] INFO RPI vc4.cpp:437 Registered camera /base/soc/i2c0mux/i2c@1/ov5647@36 to Unicam device /dev/media2 and ISP device /dev/media0
[0:39:10.765746440] [1610] INFO RPI pipeline_base.cpp:1101 Using configuration file '/usr/share/libcamera/pipeline/rpi/vc4/rpi_apps.yaml'
[0:39:10.779440096] [1609] INFO Camera camera.cpp:1033 configuring streams: (0) 1296x972-XBGR8888 (1) 1296x972-SGBRG10_CSI2P
[0:39:10.780266597] [1610] INFO RPI vc4.cpp:565 Sensor: /base/soc/i2c0mux/i2c@1/ov5647@36 - Selected sensor format: 1296x972-SGBRG10_1X10 - Selected unicam format: 1296x972-pGAA
Capturing..
0 time: 0.12585759162902832
1 time: 0.06535124778747559
2 time: 0.0605008602142334
3 time: 0.06431961059570312
4 time: 0.06484651565551758
5 time: 0.06189727783203125
6 time: 0.06329607963562012
7 time: 0.06274724006652832
8 time: 0.06275367736816406
9 time: 0.0623621940612793
14.327 fps, 10 images
Saving..(total 10 images)
Finished
- Capture BMP, save JPG faster fps (~14 fps), good file size (~90KB)
# camera.capture_file(data, format='bmp')
# byte_image.save("file_jpg"+str(k).zfill(2)+".jpg", "JPEG")
# FPS: 14.327
# frame capture time: ~62ms
# write speed: ok
[0:45:49.031581865] [1645] INFO Camera camera_manager.cpp:297 libcamera v0.0.5+83-bde9b04f
[0:45:49.079437062] [1647] INFO RPI vc4.cpp:437 Registered camera /base/soc/i2c0mux/i2c@1/ov5647@36 to Unicam device /dev/media2 and ISP device /dev/media0
[0:45:49.079620810] [1647] INFO RPI pipeline_base.cpp:1101 Using configuration file '/usr/share/libcamera/pipeline/rpi/vc4/rpi_apps.yaml'
[0:45:49.093474390] [1645] INFO Camera camera.cpp:1033 configuring streams: (0) 1296x972-XBGR8888 (1) 1296x972-SGBRG10_CSI2P
[0:45:49.094166986] [1647] INFO RPI vc4.cpp:565 Sensor: /base/soc/i2c0mux/i2c@1/ov5647@36 - Selected sensor format: 1296x972-SGBRG10_1X10 - Selected unicam format: 1296x972-pGAA
Capturing..
0 time: 0.12216925621032715
1 time: 0.06516504287719727
2 time: 0.062499284744262695
3 time: 0.06568217277526855
4 time: 0.06834888458251953
5 time: 0.06104230880737305
6 time: 0.06442689895629883
7 time: 0.0646357536315918
8 time: 0.06180119514465332
9 time: 0.06397366523742676
14.224 fps, 10 images
Saving..(total 10 images)
Finished
We also read a bit about capturing MJPEG video since it can somehow be divided again into separate JPG frames. However, there were some notes about image quality, and it still does not give control on triggering a flash at the start of each frame. We understand there are many things happening in the background in the camera module itself, which was highlighted on one of the posts on stackoverflow or stackexchange (we are not able to find it now though), but there should be a way to know when a frame starts, or if it is captured and ready, to be able to trigger flash or do external processing.
