-
Notifications
You must be signed in to change notification settings - Fork 3.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cosmos: EF Core 9 fails to find document with '|' character in it's id #35224
Comments
@mvaisanen In EF9, we changed the way we generate key values. To get back to the old behavior, you can use: modelBuilder.HasDiscriminatorInJsonIds(); See The id property now contains only the EF key property by default for more information. Notes for team: EF appears to be doing the correct thing when creating the documents: {
"id": "Cat|1",
"Discriminator": "Cat",
"Name": "First Cat",
"_rid": "59QFAK-3p3YBAAAAAAAAAA==",
"_self": "dbs/59QFAA==/colls/59QFAK-3p3Y=/docs/59QFAK-3p3YBAAAAAAAAAA==/",
"_etag": "\"00000000-0000-0000-4275-0430696601db\"",
"_attachments": "attachments/",
"_ts": 1732894828
}
{
"id": "Cat2",
"Discriminator": "Cat",
"Name": "Second Cat",
"_rid": "59QFAK-3p3YCAAAAAAAAAA==",
"_self": "dbs/59QFAA==/colls/59QFAK-3p3Y=/docs/59QFAK-3p3YCAAAAAAAAAA==/",
"_etag": "\"00000000-0000-0000-4275-0436508d01db\"",
"_attachments": "attachments/",
"_ts": 1732894828
} Notice that there is a single However, when we attempt to perform a
Both the Cosmos ID and the Cosmos partition key should be |
Also, note for team: I'm not marking this as a regression, since the breaking change was intentional, and when the API to revert back to the old behavior is used, then everything works still. |
@ajcvickers but if we generate the wrong id for ReadItem, doesn't it mean that there are queries which worked before and which don't now (returning no results)? |
@roji Yes, that's what this issue is. But we knew this was going to happen because we create key values differently--this is the breaking change. So I don't think we should treat it as a regression because of that. But honestly, I expect the fix will not be high risk, and I'm fine with shipping any low-risk patch. Tactics might not be. |
OK, understood! |
Fixes #35224 When we generate a key containing multiple values, then we escape the separator character if it is found in any of the values. However, we should not do this if the key is made up of a single value and so will have no separators.
…t/efcore#35264. Changed to all Async methods, added support for json delimiters. Unit tests passing. Updated readme.md.
I'm working on a project which for legacy reasons have lots of documents with the '|' character in their id. To be precise, these used to be '/' in another database and since cosmos doesnt support those, they were converted to '|'. There needs to be some special character in the id as parts of business logic relies on that (and converting the id to base64 would make other matters difficult).
Up until EF Core 9 they worked fine (EF Core 8.0.11 works). However, when upgrading to EF Core 9, these documents cannot be found by the framework. Here is a small .NET 8 console app demonstrating the issue. Example run against Cosmos DB Emulator database "TestDatabase", Autoscale max 4000 RU/s, having 1 container "TestContainer" with partition key "/id".
Program.cs
PetContext.cs
Pet.cs
.csproj
When running the app, second cat is found as expected, but query against the first fails with "Sequence contains no elements". Doesnt work with FindAsync() etc either. Examining the logs, it seems EF Core inserts an extra '^' character to the id of the first cat query, which explains why it fails on db level.
I'm aware of the recommendation to use only alphanumeric ids (https://learn.microsoft.com/en-us/azure/cosmos-db/concepts-limits), however this used to work ok on earlier versions, and still works just fine when running manual queries against cosmos. The issue seems to be just that EF Core corrupts the id intended to be queried. However, as the example shows, even saving works just fine.
Provider and version information
EF Core version: 9.0.0
Database provider: Microsoft.EntityFrameworkCore.Cosmos
Target framework: NET 8.0
Operating system: Win10 Enterprise
IDE: Visual Studio 2022 17.9.4
The text was updated successfully, but these errors were encountered: