Myself, Ashu. Very happy to have a python library solely focused on traditional CV. I wanted to ask about Zernike moments (ZMs) as feature descriptors. As far as I know, there is no implementation in the skimage library. I have searched ‘zernike’ keyword on the website, in skimage forum, Zulip chat, issues on github, my recent forked repo of skimage. Couldn’t find any mentions or results. Is it being implemented currently, or maybe skimage2? Was it planned in the past then dropped from the library for some reason?
ZMs are used extensively in optical systems for aberration corrections, where I learned these. I have read the issue #4375 (Show and tell) on GH and see people using skimage for microscopy, shape, defect analysis etc. Assuming there is a need and it’s not being developed, will the community be interested in Zernike moments as a new library feature? If so, I would like to develop it, or help if someone is developing it.
Zernike moments refresher: Given an image, a unit circle centered over the image/object (circular pupil in optical systems, circular bounding box in object detection), Zernike moments are image weighted average of Zernike polynomials. There are three terms here: polynomials are complex valued basis functions which iterate over frequencies (cos(1,2,..,Mθ)); moments are image/object pixel intensities weighted average of these polynomials; features are magnitude of these complex moments. Degree n is a design parameter for the user, and fitting a degree n polynomial gives a feature vector of length (n+1)*(n+2)/2. So a degree=9 will give to the user a feature vector of length 55 with values between 0 and 1 (normalized).
For development and library features, and assuming not being developed by someone, I have the following in mind:
Develop the base Zernike moments (zernike.py) using pure python. This is to familiarize myself with the skimage contribution standards and procedures. I will implement feature, interface, documentation, references, tests, examples etc., then generate a PR. Please critique any minor to major changes. For the math heavy work, I want to ask if someone can verify the implementation.
Develop the cython extension for faster computation. Here I will need help from the core team as I am not quite familiar with cython and skimage build process. I will provide a working extension (zernike_cy.pyx), but will need help optimizing it. If we imagine a user iterating through thousands of images/objects, the aim is to compute the features very fast (<1s per image/object).
Once the base Zernike is optimized and finished, I have 4/5 more Zernike related library features in mind. If interested, I will build these and we can gradually roll out.
Please let me know if this would be something community is interested in, how would you like to proceed, your thoughts on development, and anything else.
A long while back I had a need for Zernike moments and I wished there was a library that had them implemented. It’s one of those things that are good to have just in case.
Hey @aacvysion, thanks for reaching out, sorry for taking so long to reply.
My knowledge regarding Zernike moments is almost non-existent, so I can’t really comment on whether we want this feature or how difficult it would be to implement. Are there already existing implementations in Python?
But I can raise your original request in our community meeting (next Tuesday).
That said: we are currently in the middle of refactoring our API to a more consistent state so our resources are very tied up right now. So you’d probably have to be pretty self-sufficient when it comes to implementing the feature. Do you already have experience with bigger contributions to open source projects?
Yeah @lagru, please let me know what will be the thoughts of the community in the meeting next week.
AFAIK, only mahotas library provides ZM implementation. There are individual repos and packages, but I don’t know which ones are popular and how they are used.
I already have a base ZM working with python, numpy, and multiprocessing. It takes ~4 secs to process one 640x640 image on my PC. I am following the contribution guidelines, so I can raise a PR in a few days with ZM module, docs, examples, tests etc. This can provide a base, so if a senior dev may want to takeover for any reason, we will have a good starting point at least.
Where I need help will be to verify the mathematics, with optimizations and Cython extension so as to make it faster, a recommended place for communication whether for minor questions or in-depth discussions (Zulip, GH PR thread etc), and all this adhering to skimage standards.@martinberoiz please let me know if you’re interested, or if anyone else in the meeting may express an interest, to work in any capacity.
Thanks for the consideration, but I forgot a lot about Zernike moments that I’m no longer confident I can do a good job on my own. I think @aacvysion can lead this and maybe I can help if help is needed, just let me know.
I have generated pull request for Zernike features. Please review it and let me know any comments, thoughts, mistakes, typos, recommendations etc. to improve the implementation.
I want to ask the community to review and verify the math behind ZFs just to be sure that everything is working as intended.
Would @martinberoiz you like to review and provide feedback in any way? Be it with math, API, advance usage, examples etc. I know about ZFs, but not how people apply it in their workflows. I plan to extend it to arbitrary pupils, RGB images etc. in the future.
Thank you @aacvysion Let me try to find someone from astronomy that could take a look in a few days, and if I don’t find anyone I can take a look myself
@aacvysion I asked around and I did not find anyone that wants to take the job of reviewing it, unfortunately. Just for the record, I’m not associated with scikit-image in any way other than a end user. I started reading the review paper you used and I wish I had read it back then! that’s a good find.
Do you still want or need a review of the PR? I don’t know if me taking a look would be of any help to have it eventually merged. I would it like to be merged. Is there anyone from scikit-image that can advice on what’s the best path to have it merged? Would my review make any difference?
Martin, I think your review would definitely help to get it merged; as Lars mentioned, we are in the middle of a big refactor, and to have new algorithmic code included it would need to be vetted. But I trust you to do that.
The best way to have things merged is to discuss it in the community calls, to get people’s attention on it. I think if you can do the algorithmic vetting, I can do the code review and we can get this included.
This will probably only become publicly available with the 2.0 release, which may take a while. But you’ll at least have it available through nightlies until then.
Hey @martinberoiz , thanks for being open and taking the time to review the math and algorithm for this. Want to give an update that I have added a latest commit in the PR that generalizes these Zernike features over 6 types of puipls. The Cython backend is simplified as well with lots of comments. Below is an example of pupils:
Thanks @stefanv for being open to code-review. I’ll drop a message in Zulip chats as well to ask the community for help with certain points.
I don’t have any feature updates planned for some time, so mostly it’ll be patching, compliance, bug-fixes, rewrites etc. for the implementation so far.
