PReLu ONNX import

[Arjun31415]

Pull Request Template

Checklist

• Confirmed that run-checks all script has been executed.
• Made sure the book is up to date with changes in this PR.

Related Issues/PRs

#1714

Changes

Added PReLu ONNX -> burn operator

Testing

Tested as specified in the Contributor book

[Arjun31415]

The rust code produced -

// Generated from ONNX "./onnx-tests/tests/prelu/prelu.onnx" by burn-import
use burn::nn::prelu::PRelu;
use burn::nn::prelu::PReluConfig;
use burn::nn::PRelu;
use burn::record::FullPrecisionSettings;
use burn::record::Recorder;
use burn::{
    module::Module,
    tensor::{backend::Backend, Tensor},
};


#[derive(Module, Debug)]
pub struct Model<B: Backend> {
    prelu1: PRelu<B>,
    phantom: core::marker::PhantomData<B>,
    device: burn::module::Ignored<B::Device>,
}


impl<B: Backend> Default for Model<B> {
    fn default() -> Self {
        Self::from_file("./out/prelu", &Default::default())
    }
}

impl<B: Backend> Model<B> {
    pub fn from_file(file: &str, device: &B::Device) -> Self {
        let record = burn::record::PrettyJsonFileRecorder::<FullPrecisionSettings>::new()
            .load(file.into(), device)
            .expect("Record file to exist.");
        Self::new(device).load_record(record)
    }
}

impl<B: Backend> Model<B> {
    #[allow(unused_variables)]
    pub fn new(device: &B::Device) -> Self {
        let prelu1 = PReluConfig::new(0, 0.009999999776482582).init(device);
        Self {
            prelu1,
            phantom: core::marker::PhantomData,
            device: burn::module::Ignored(device.clone()),
        }
    }

    #[allow(clippy::let_and_return, clippy::approx_constant)]
    pub fn forward(&self, input1: Tensor<B, 2>) -> Tensor<B, 0> {
        let prelu1_out1 = self.prelu1.forward(input1);
        prelu1_out1
    }
}

I think the PReluConfig initialization is wrong .

        let prelu1 = PReluConfig::new(0, 0.009999999776482582).init(device);

the number of parameters should not be 0 and initial alpha should be 0.01 exactly and not this approximate value

[antimora]

Great start but we are missing two types of unit tests (under onnx-tests and in crates/burn-import/src/burn/node/prelu.rs). Also Prelu ONNX spec does not have alpha and numb_parameters, so there is no need to extract configuration.

[antimora]

FYI, to run ONNX test:

[burn-import]% cd onnx-tests
[onnx-tests]% cargo test

to run code gen tests:

[burn-import]% cargo test

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 86.47%. Comparing base (ab50143) to head (9e851b4).
Report is 8 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1721      +/-   ##
==========================================
- Coverage   86.54%   86.47%   -0.07%     
==========================================
  Files         696      698       +2     
  Lines       81653    82874    +1221     
==========================================
+ Hits        70664    71667    +1003     
- Misses      10989    11207     +218     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[Arjun31415]

I also get this warning

DEBUG burn_import::onnx::from_onnx: Number of initializers: 1    
DEBUG burn_import::onnx::from_onnx: Number of outputs: 1    
DEBUG burn_import::onnx::proto_conversion: Converting ONNX node with type "PRelu"    
DEBUG burn_import::onnx::from_onnx: renaming node "/relu1/PRelu"    
 WARN burn_import::onnx::dim_inference: Must implement dimension inference for PRelu    
DEBUG burn_import::onnx::from_onnx: checking inputs for node "prelu1"    
DEBUG burn_import::onnx::from_onnx: 

[antimora]

I also get this warning

DEBUG burn_import::onnx::from_onnx: Number of initializers: 1    
DEBUG burn_import::onnx::from_onnx: Number of outputs: 1    
DEBUG burn_import::onnx::proto_conversion: Converting ONNX node with type "PRelu"    
DEBUG burn_import::onnx::from_onnx: renaming node "/relu1/PRelu"    
 WARN burn_import::onnx::dim_inference: Must implement dimension inference for PRelu    
DEBUG burn_import::onnx::from_onnx: checking inputs for node "prelu1"    
DEBUG burn_import::onnx::from_onnx: 

Yeah, you will need to add dim inference in dim_inference.rs file. You can use same as input function since the output will have the same dimensions.

[antimora]

Also please update the supported https://github.com/tracel-ai/burn/blob/main/crates/burn-import/SUPPORTED-ONNX-OPS.md as well

[Arjun31415]

Ok so I think what is pending is the dim_inference and loading the weight from the onnx into alpha of the Burn PreluConfig. I need help with this, where exactly do I do the latter part

[Arjun31415]

if I do those two then tests would also be fixed

[Arjun31415]

Ok I think dim inference is also fixed. Now I just need to load the prelu weight into burn

[Arjun31415]

Ok I think it is done

[antimora]

Looks great!

Thank you for your contribution to the Burn project!