IMX477 Improvements - 6.12 #6798
Conversation
|
Hi, after fixing my noob setup i can confirm its working, meanwhile are we expecting to merge this soon? |
|
Rebased which will retrigger the CI builds. My comment from #6208 (comment) that 2028x1080 and 2028x1520 modes with 10 bit readout give totally white frames needs to be resolved before contemplating merging. |
|
I have done some testing with this, yes the 10bit 2K modes are white frames |
6by9
force-pushed
the
rpi-6.12.y-imx477
branch
from
576c474 to
579b122
Compare
|
I've finally had a few minutes to relook at this one. I've dropped the patch that added 10 and 12 bit readout to all modes. That can be reintroduced in due course. What I have done is reworked setting the link frequency so that it will accept any value that can be achieved (basically a multiple of 3MHz). At 750MHz we get It's not quite right as the base 450MHz drops performance on the 1332x990 mode. My 500usecs for LP<>HS is probably too generous, and I'm not accounting for it only being a 10bit mode. I haven't seen any image corruption issues, but it hasn't had masses of testing. It would be appreciated if those interested would test it and feed back any issues. |
|
Some testing ** (well beyond the suggested max of 750 MHz) showed only occasional frame dropping in any of the modes up to the point where I stopped at 996 Mhz. At that frequency the full frame mode was exceeding 25 fps. However somewhere between 924 and 948 MHz both of the 2x2 binned modes started outputting solid colour frames. Magenta this time rather than white as in the previous 10 bit issue. |
|
Thanks for testing. As this is a sensor driver, I'm not that interested in anything above 750MHz as then you're exceeding the Pi's capabilities rather than the sensor's. Pi5's libcamera pipeline handler already knows that RP1 CFE has a limit on pixel rate (250MPix/s IIRC), so I'm expecting that will be automatically extending the HBLANK control value once you get to a point where the link frequency can exceed the limit, but you won't get any increase in output framerate from libcamera on Pi5. |
There should be more headroom than 250Mpix/s, ~380Mpix/s is the standard / supported pixel rate. Even at the higher link frequencies tested by @sandyol55, I never saw any frame size/frame rate combo that exceeds 300Mpix/s. Here is 945Mhz So there must be something else at play that is causing those white frames? It's just so strange that it would be isolated to the 2x2 binned modes, even at low frame rates, because the full readout/4k modes operate just fine from what I've seen; and those ones do in fact operate at pixel rates in many cases approaching 300Mpix/s, which are still well below the 380Mpix/s limit. |
|
@schoolpost Are you managing to record ok the uncropped 4056x3040/24 modes? Curious how you're managing that if so. Testing with Cinemate, the preview looks good but as soon as I record the preview drops frames making these modes unusable for me, it isn't dropping frames on the disk but at the camera. |
I don't believe that would a be driver related issue, more so application level and buffer management ( recycling buffers in a timely manner ) |
|
You're right - it's 380MPix/s that libcamera limits the pixel rate to. My memory is going! Greater than 750MHz is still above RP1's spec, so it's still debatable as to whether it is the sensor setup or Pi misbehaving. I haven't confirmed that the PLL configuration is still within the specs. Spec says the max multiplier value is 350. The pixel rate PLL is always set to 840MHz, and that is sufficient for full res @60fps, so that shouldn't be a limit. Just as long as we set LINE_LENGTH_PCK to a sufficient value to be able to transmit all the data. |
6by9
force-pushed
the
rpi-6.12.y-imx477
branch
2 times, most recently
from
0d3748c to
0aa0f94
Compare
|
I'm happy with these now. At 750MHz we get On my module 831MHz is also stable and means the 4056x2160 mode can achieve 30fps. That's about the limit of where I'm prepared to say things are supported on the Pi. We have noticed an apparent gain oscillation when going from dark to light scenes at high frame rate, however that behaviour can also be reproduced with the old driver. I'm handing that one over to @naushir to investigate. |
|
Hang on, register 0x0820-0x0823 are meant to be REQ_LINK__BIT_RATE_MBPS, which we aren't being changed. |
6by9
force-pushed
the
rpi-6.12.y-imx477
branch
from
0aa0f94 to
b3065f6
Compare
|
REQ_LINK__BIT_RATE_MBPS and Global MIPI Timings are now set sensibly, so I really am stopping fiddling now! |
6by9
force-pushed
the
rpi-6.12.y-imx477
branch
from
b3065f6 to
037368b
Compare
|
As alluded to by @6by9, no changes in the white frame issue with the latest changes. ( I did notice some higher frame rates for the 10bit "990p" mode ) Would be great to have the mirrored sensor modes for both 10bit and 12bit as we did at some point earlier in these changes. I will note that the exact point in which we go from valid frames to white frames occurs at the boundary of 939Mhz and 942Mhz ( this was both with this newer driver from today and the one from a few days ago ), I don't know if there some magic number component to this; maybe that provides some much needed context to why this is happening in the first place? I wish I could get a better idea, but without any reference to the datasheet/registers it's just a big guessing game... Also strange that "990p" mode is unaffected by higher link frequency errors. only the other 2 binned modes.... Does some seem like everything else works, Thanks again @6by9 |
That sounds promising, so you think this is something which needs attention not in cinepi-raw but in cinemate? |
|
I may resurrect supporting 10 and 12 bit readout for all modes at a future date, but not now. Seeing as we are now computing the hblank minimum based on link frequency, bpp, width, etc, it may just fall out, but I'm meant to be working on other things. Rereading the datasheet, whilst the A/D supports 10 or 12 bit conversion, there's the option of an 8 bit output image too. That should be simple to add too. The white frames on the binned modes are likely something in the sensor. My module works at 939MHz, but fails at 942MHz. |
Register line_length_pix was being written by both the tables of registers and the control handler for V4L2_CID_HBLANK. Remove the duplication in the tables. Signed-off-by: Dave Stevenson <[email protected]>
line_length_pix is a value that the developer wants to know, so write the values in decimal. Signed-off-by: Dave Stevenson <[email protected]>
The frame length default value doesn't change dynamically, and neither does any of the other parameters that configure it, so precompute it instead of working from a frame duration to get to the value. The minimum value was also computed, when actually the sensor will take any value down to 4 lines. Signed-off-by: Dave Stevenson <[email protected]>
This removes a load of boilerplate code around how registers are grouped into multiple word values. Signed-off-by: Dave Stevenson <[email protected]>
There are a fair number of registers duplicated in all the mode tables, so move those into the common table. Signed-off-by: Dave Stevenson <[email protected]>
For 4k30 recording we want 16:9 output, so add a cropped mode to achieve this. Signed-off-by: Dave Stevenson <[email protected]>
Rather than the hard coded PLL settings for fixed frequencies, compute the PLL settings based on device tree, validating that the specified rate can be achieved. Signed-off-by: Dave Stevenson <[email protected]>
As we now support variable link frequency, compute the minimum line_length value that the sensor will work with, and set V4L2_CID_HBLANK based on that number. Signed-off-by: Dave Stevenson <[email protected]>
Now that the link frequency can be varied, write the link bit rate registers to reflect the speed being used. Signed-off-by: Dave Stevenson <[email protected]>
The timing registers configured are for 450MHz. If running at a different link frequency, use the automatic timing control. Signed-off-by: Dave Stevenson <[email protected]>
The sensor supports readout as 10 or 12 bit. As we are now computing the horizontal blanking limits dynamically, adding support for both readout modes falls out trivially, so add them both. Signed-off-by: Dave Stevenson <[email protected]>
6by9
force-pushed
the
rpi-6.12.y-imx477
branch
from
037368b to
41a7fe6
Compare
8 bit readout is only a reconfiguration of the CSI2 block, and recomputation of horizontal blanking. Enable it. Signed-off-by: Dave Stevenson <[email protected]>
I really shouldn't get distracted. I've done the mods for 10 or 12 bit readout in all modes as it was effectively cherry-picking an existing commit. At 450MHz, all modes work at either bit-depth. At 750MHz, 4056x3040, 4056x2160, 2028x1520, and 2028x1080 are fine in either bit-depth. 8bit readout had resulted in magenta images in some modes, but retesting and I can't reproduce it. Ho hum. I've pushed the patch that adds it. 450MHz |
|
Hats off to 6by9 for allowing some distraction!! These updated capabilities could give the HQ sensor a whole new range of potential applications. On my system, testing link frequencies up to 909 MHz gave all 15 modes operating with no corrupted frames, and no fixed colour frames in any mode. At 912MHz, and above, the 8-bit 1332x990 mode -(and only that mode)- showed some magenta-tinted break-up in the lower half of the image. This pattern continued up to a maximum tested frequency of 1050MHZ where all 10-bit and 12-bit modes were still OK, as were the remaining 4 full width 8-bit modes. Reported capabilities as below with a standard IMX219 V2 camera for comparison. I note that the 4 full-width 8-bit modes report the same fps performance at both 909 and 1050 MHz link frequencies, while the cropped 1332x990 mode reports slightly different values, 285.47 v 287.03 MHz. |
|
I'm done then. Over to @naushir to review. |
|
A stupid question - once this get merged, all these modes will be available in the standard 6.12 kernel, right? Will it still require dtoverlay to enable? |
8, 10, 12 bit readout for all modes will be available as standard even when the firmware autodetection is used. The default link frequency will remain at 450MHz, giving which is a small bump in most modes. At least initially increasing the link frequency will require a manual I'll discuss with colleagues as to whether it is worth modding the Pi5 bootloader to automatically add an override to 750MHz for autodetected cameras, thereby increasing the default capabilities. |
|
Awesome, thanks for the quick answer! Eagerly waiting for the merge then ^_^ One more thing, just to get a reality check. Do I understand it correctly that encoding at 2160p in H.264 in real-time is unachievable even on Pi 5? From what I read on pi forums - CPU just won't be able to keep up... Is MJPEG the only viable option? (and I guess RAW dumping should work too) Not planning to start a discussion here, just wanted to understand the situation with real-world application of these new capabilities :-) |
2160p frames can be encoded as H264, but the frame rate achievable will depend on encoding options. Definitely no B-frames, potentially 1 reference frame, bitrate may be limited due to the complexities of CABAC. |
6.12 version of #6208
Only compile tested.