AVRO-2825: [csharp] Resolve: C# Logical Types throw exception on unknown logical type #2741
Conversation
Language implementations must ignore unknown logical types when reading, and should use the underlying Avro type.
also resolved exception thrown if namespace is missing from schema
… type" This reverts commit c9c1bed.
… the schema file does not contain a namespace This reverts commit 3f75a9a.
| try | ||
| { | ||
| if (null != jo["logicalType"]) // logical type based on a primitive | ||
| return LogicalSchema.NewInstance(jtok, props, names, encspace); | ||
| } | ||
| // swallow exception from unknown logicalType | ||
| catch { } | ||
|
|
There was a problem hiding this comment.
Can LogicalSchema.NewInstance instead be made to work with unrecognized logical types? For these purposes:
- Avoid swallowing any exceptions thrown for other reasons.
- Allow applications to parse a schema and read the name of the logical type from LogicalSchema.LogicalTypeName (or even LogicalType.Name) regardless of whether the library recognizes it.
- Allow applications to parse a schema and serialize it back to JSON without losing the unrecognized logical types here:
avro/lang/csharp/src/apache/main/Schema/Property.cs
Lines 45 to 46 in 49e619b
LogicalSchema.LogicalType could then be null, or perhaps an instance of a new NotSupportedLogicalType class that would pass everything through without conversion.
There was a problem hiding this comment.
I will work on that, thanks for the feedback
| if (string.IsNullOrEmpty(nspace)) | ||
| { | ||
| throw new CodeGenException("Namespace required for record schema " + recordSchema.Name); | ||
| } | ||
|
|
||
| CodeNamespace codens = AddNamespace(nspace); | ||
|
|
||
| //if (string.IsNullOrEmpty(nspace)) | ||
| //{ | ||
| // throw new CodeGenException("Namespace required for record schema " + recordSchema.Name); | ||
| //} | ||
|
|
||
| // AVRO spec DOES NOT require a Namespace but this code does. | ||
| // Workaround is to inject a fixed string that will be obvious if required | ||
| CodeNamespace codens = (!string.IsNullOrEmpty(nspace)) ? AddNamespace(nspace) : AddNamespace(@"SchemaHadNoNamespace"); |
There was a problem hiding this comment.
This is not related to logical types. Please move to a separate pull request.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| return LogicalSchema.NewInstance(jtok, props, names, encspace); | ||
| } | ||
| // swallow exception from unknown logicalType | ||
| catch { } |
Check notice
Code scanning / CodeQL
Generic catch clause Note
| return LogicalSchema.NewInstance(jtok, props, names, encspace); | ||
| } | ||
| // swallow exception from unknown logicalType | ||
| catch { } |
Check notice
Code scanning / CodeQL
Poor error handling: empty catch block Note
| var secondField = ((RecordSchema)schema).Fields.FirstOrDefault(f => f.Name == @"secondField"); | ||
| Assert.IsNotNull(secondField); | ||
|
|
||
| var secondFieldSchema = ((Field)secondField).Schema; |
Check warning
Code scanning / CodeQL
Cast to same type Warning test
AVRO-2825
What is the purpose of the change
NOTE: This PR WILL change the behavior of the current nuget package. It corrects it to align with the AVRO spec.
Verifying this change
This change is already covered by existing tests:
test > AvroGen > AvroGenSchemaTests.cs > NotSupportedSchema
test > Schema > SchemaTests.cs > TestUnknownLogical
This change added tests and can be verified as follows:
Documentation