Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add CAN model #739

Closed

Conversation

zhangjunlongtech
Copy link
Contributor

Thank you for your contribution to the MindOCR repo.
Before submitting this PR, please make sure:

Motivation

  • MindOCR does not currently support handwritten mathematical formula recognition. I hope to solve this problem by contributing a CAN model.

  • HMER(Handwritten Mathematical Expression Recognition) mostly uses the encoder-decoder mechanism. However, when identifying long or complex formulas, it cannot guarantee the accuracy of the region of interest of the Attention module. CAN(Counting-Aware Network) utilizes Multi-Scale Counting Module to improve the accuracy of formula recognition, by introducing counting vectors that can provide global information and spatial position codes that can provide position information.

  • The CAN model consists of the backbone and head modules. For details about the backbone module, see rec_densenet.py. For details about the head module, see rec_can_head.py.

Test Plan

  • The model unit test file test_can_model has been provided. This file sets model parameters and test case parameters.
  • The test file tests the BaseModel class in mindocr/models/base_model.py. The shape of the tensor in the test result meets the expectation.

image

  • To verify the output of backbone, the build_backbone method in mindocr/models/backbones/builder.py is tested based on rec_densenet in CAN model. The tested tensor shape meets the requirements of the paper.

image

  • Tested build_head method in mindocr/models/heads/builder.py based on backbone's output.

image

Related Issues and PRs

Related Issues: Model CAN Model Support

from mindspore import nn
from mindspore import ops

ms.set_context(mode=ms.PYNATIVE_MODE, pynative_synchronize=True)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why set pynative mode in model definition file?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it can change the global context when imported

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants