Contributing new algorithms to skimage (PhyCV by UCLA, no patent)

Contributor & Development Discussion
1

Hey everyone,

Researchers from the Jalali Lab at UCLA are looking to contribute some new computer vision algorithms to scikit-image! We’ve developed these as part of our PhyCV project (GitHub - JalaliLabUCLA/phycv: PhyCV: The First Physics-inspired Computer Vision Library) and we think they could really benefit the community.

Here’s the lowdown:

Both VLight and PhyTex are fully open-source and patent-free. We’ve got them implemented in Python, ready to go for scikit-image.

We believe these algorithms would be awesome additions to the library, providing users with cutting-edge tools for image processing and analysis. We’re more than happy to work with the scikit-image team to ensure smooth integration and documentation.

Let us know what you think!

Best,

Wesley Gunawan Jalali Lab @ UCLA

2

Hi All,

I’ve pushed an implementation of VLight PhyCV to the following branch:

If you see any areas for improvement, please feel free to modify the implementation and communicate your feedback or updates here in this forum. Your input would be greatly appreciated!

I’m also reaching out to the skimage core team members to discuss the possibility of pushing this to the main branch.

Thank you in advance for your time and contributions!

Best regards,
Wesley

3

Hi Wesley,

Thanks for your interest in contributing to skimage!

Is the function proposed here different from Gamma correction?

Best regards,
Stéfan

4

Hi Stéfan,

My name is Callen, I am Wesley’s colleague and will be following up for him from here.

The function proposed here differs substantially from Gamma correction, as it’s derived from a physics-inspired approach and outperforms Gamma under extreme low-light conditions, as documented in [1]. Specifically, it exploits the natural equalization property inherent in coherent detection receivers and represents the newest technique in the Physics-inspired Computer Vision (PhyCV) family of algorithms developed over the last decade [2].

Three key differences from Gamma correction are:

Physical and Mathematical Approach:

  • The new function, called VLight, is derived from physical principles of spectral phase modulation and coherent detection. This process results in a natural dynamic range enhancement of the original image [1]. Broadly speaking, the algorithm exploits the natural advantages of phase/frequency modulation and phase sensitive detection that underpins radio communication.
  • It combines equalization (for boosting dark regions), compression (to manage dynamic range), and normalization — all derived from optical physics principles.
  • Gamma correction simply applies a purely empirical function I_out = I_in^γ.

Adaptive Behavior:

  • VLight’s parameter v dynamically controls both regularization and enhancement together, allowing for more nuanced adjustments.
  • Gamma correction’s γ parameter has a more basic and direct effect.

Advantages over Gamma correction:

  • The new function exhibits better handling of extreme dark regions without amplifying noise which would occur when Gamma correction is applied to such regions.
  • As shown in [1] the results more closely resemble the normal-light groundtruth in terms of both visual appearance and histogram.
  • Gamma Correction tends to shift pixel values towards higher intensity regions without adequately preserving contrast. In other words, it distorts the histogram. The new algorithms better preserves the fidelity of the histogram [1].
  • The new function has superior quantitative performance (PSNR, SSIM) compared to Gamma correction across multiple datasets, as shown in rigorous benchmarks in [1]. A demonstrative result can be found here: Table 3 | Real-time low-light video enhancement on smartphones | Journal of Real-Time Image Processing.

In summary, the new algorithm is a more sophisticated approach that combines the physical intuition of optical processes with the speed advantages of modern computational optimization. Given the novel origin of the algorithm and its superior performance for extreme low light conditions, it will widely benefit the skimage community by offering an alternative to traditional methods like Gamma correction.

If I can provide any additional information or a rigorous mathematical derivation, please let me know.

Best,
Callen

References:

[1] Zhou, Y., et al. “Real-time low-light video enhancement on smartphones.” Journal of Real-Time Image Processing 21.5 (2024): 155.

[2] Zhou, Y., et al. “PhyCV: the first physics-inspired computer vision library.” arXiv preprint arXiv:2301.12531 (2023).

5

Hi Stéfan,

I hope this message finds you well. I wanted to revive the discussion regarding the proposed VLight algorithm for low-light image enhancement. We previously explained the fundamental differences between VLight and traditional Gamma Correction.

We believe VLight could add significant value to the skimage library, especially for users looking for robust low-light enhancement tools.

Could anyone help us with a detailed review of VLight or suggest further improvements so that we can merge it with the main repository? Your feedback would be invaluable in refining and aligning our algorithm with the community’s expectations.

Best,
Callen