[Question]: Same item found in different lists - how can I idiomatically solve the "shared state" problem?

[lucavenir]

In a Flutter application, we have different paginated lists which can possibly overlap, i.e. an Item could appear in myItems(page: 2), favoriteItems(page: 0), searchedItems(page: 7), ...
We want a idiomatic, testable and clean way to synchronize shared state (e.g. isFavorite state) for all of the above without adding extra BE requests.

There's a complete example available to reproduce the question above.

EDIT. Originally, this issue proposed an imperative API, which is rejected by Riverpod's declarative nature. The original proposal has been edited away from this issue.

[rrousselGit]

The main takeaway is: there is no easy way to pass in state from a provider to its "child" (sort-to-say).

What does this mean exactly?
Isn't that what providers do, passing state to their children?

[rrousselGit]

Solving (1) would look as easy as:

@riverpod
class IsFavoriteController extends _$IsFavoriteController {
  @override
  int build(int id) => throw UnimplementedError('This provider is meant to be overridden');
}

@riverpod
FutureOr<List<Item>> someList(SomeListRef ref) async {
  final result = fetch(fromSomewhere);
  for (final item in result.items) {
    ref.override(isFavoriteControllerProvider(id), w: (ref) => item.isFavorite);
  }
}

Why can't you do:

class IsFavoriteController extends _$IsFavoriteController {
  @override
   bool build(int id) => ref.watch(someListProvider).items.firstWhereOrNull((i) => i.id == id)?.isFavorite ?? false;
}

[rrousselGit]

Honestly there's a bit too much to unpack in that issue.
It's not immediately clear to me what we're trying to solve. It's hard to redirect you to another solution or issue as I'm not fully sure what this is about. It'd help if you could give smaller examples than the repo you linked. Your example has way too much code for me to know what exactly you're trying to showcase here.

But no matter what, imperative overrides are a no-go.
No matter whether we would want those, we cannot have those. It is quite fundamental to how Riverpod works that overrides are fully static.

There's a reason why ProviderContainer/ProviderScope don't enable adding/removing new overrides. It doesn't work :)

[lucavenir]

Honestly there's a bit too much to unpack in that issue.

Okay - true - give me one more chance - let's focus just on part (1) for now if you will

Why can't you do [...]

I forgot to mention that someListProvider is paginated - every list in these example is, even the favorite one.

1. what's the correct page to watch? Should I pass page around everywhere - which is not really feasible for my use case. (page leaks everywhere - it not just tedious but... you quickly find yourself with widgets or provider needing a page they don't really care about)
2. accessing lists is generally unsafe - especially the ?? false part: you can never really rely on firstWhere, e.g. invalidation refresh the provider and removes an item, or any other unexpected change. Generally speaking, the mental reactive model I have should watch this detail and not the other way around

It's not immediately clear to me what we're trying to solve.

I'm trying my best to summarize my use case, but without the right constraints, riverpod has a solution for the above problem. With the right requirements, I find myself in a pinch and I'm not sure this issue is open in here.
Give me one more chance with a bullet point:

• You want to handle several list of items, paginated, possibly infinite, that have some common properties that need to be in sync!
• I can't find an idiomatic way to "always find app state in sync between multiple providers (or controllers) that reference the same items (semantically, e.g. same id)"
• "in sync" includes and isn't limited to the following events: some items are removed from the favorites (they disappear from a list and their favorite status should be propagated everywhere), some items are added from the favorites (they appear in the favorite list, propagate it), some items are deleted entirely (etc.)

Your example has way too much code

Yes. Sorry.
Don't look at the code - execute the trivial app and notice how an extra request is performed when you tap a like button, then - if you may - read the summary above again to understand why I must perform that to synchronize state everywhere.

But no matter what, imperative overrides are a no-go.

Honestly I'm fine with whatever. Procedural, Functional, Object-Oriented, Event-based. Fine.
I need to ship, ship, ship features. Test them. And "fast" (aka at the right speed).
I'm all-in for reactivity - I guess you know 😛 - but I can't solve this problem.
Maybe a reactive emit will do (aka let provider register callback listeners that are triggered on emits from below the tree)? I'm not sure. I'm all ears

[rrousselGit]

1. what's the correct page to watch? Should I pass page around everywhere

Sure, pass it around if need be.
If may be inconvenient by your standards, but it works.
I know that you dislike props drilling, as we've discussed variants of this multiple times. But that's ultimately the current solution.

---

I'm open to proposals. But I don't think this specific feature request is realistic for technical reasons.
If you have other ideas, I'm open to investigate those.

Although you always have the possibility to make a Notifier with a public setter in it, and invoke that setter wherever you want.
You'll break a few Riverpod principles and may have to use workarounds to bypass the internal error handling, but that should work.

[lucavenir]

Hi, I'm back. I wanted to reason about and experiment on this before wasting your time.

Sure, pass it around if need be.

I don't mind passing around props anymore, since our last discussion I accepted that 😄 I tried doing that, but it doesn't solve the problem.

You proposed:

class IsFavoriteController extends _$IsFavoriteController {
  @override
   bool build(int id, int page) => ref.watch(someListProvider(page)).items.firstWhereOrNull((i) => i.id == id)?.isFavorite ?? false;
}

But this won't work, because:

1. an item could be in searchProvider(page: 5), into myFavoritesProvider(page: 0), and into myListProvider(page: 1). See? Passing page around gives inconsistent states, because .firstWhereOrNull will return null in most cases, thus false even if it's favorited
2. say we solve (1), somehow. Again, we have myListProvider, myFavoritesProvider, searchProvider, homeProvider, anotherListProvider. They're all asynchronous List<T> fetchers. Question: which of these should IsFavoriteController watch?

Then you also propose to use a Notifier just to have a public setter in it, but as you've pointed out that clearly breaks riverpod usage. I also am not sure how this Notifier should be initialized (null? yuck!)

As you've rightfully pointed out, there're many things to unpack.
Therefore, can we narrow this down to solving this "shared state" problem?

I am not able to understand how riverpod can help me here. Shared state is exactly why I'm adopting riverpod, hence this issue. So, thank you for replying up until now (:

[lucavenir]

I also changed the first comment so this issue is narrowed down to a clear and simple problem. I hope this helps.

[mattermoran]

This applies to essentially any app out there. And it's very common to see inconsistent state in the app because it's changed in one place but not other (especially inside lists).

The way I handle it is by essentially having a single provider per Item. And instead of rendering List<Item> you render List<Reference<Item>> so the list is always up to date. In this case my whole app is synchronized. Doesn't matter if I have 100 different lists where that item is present. They all will render exactly same thing. And once one of the properties is updated like isFavorite because button was clicked or just because new state comes in a new list that will be reflected everywhere.

As to how exactly to do this is up to you.

In my case I use graphql, hook into response coming from api, and then place it into my cache that is then read by a provider.

This is a very basic explanation as I do bunch of other stuff like normalizing data to handle deeply nested objects, merging data so incompletely data doesn't just override, etc.

The comes with bunch of other advantages like navigating around the app with just ids as you always know where to get that data from, "free" offline mode as you already have the data cached, etc

But you can keep is simple and skip normalization if you have simple enough app and use-case.

I'm sure I can make it into a package. It's very generic and technically could be used with any "provider" package out there be it riverpod, bloc, or anything

[rrousselGit]

Note that 3.0 is already adding a way to listen a provider without mounting it:

ref.listen(
  provider,
  weak: true,
  (prev, next) {

});

This should help solving this sort of problem. An Item provider could listen to multiple providers emitting an Item instance and pick the most recent one, without triggering those providers.

So you could do:

@riverpod
Future<List<Item>> home() => get('/home');

@riverpod
Future<Item> byId(Ref ref, {required String id}) {
  ref.listen(
    homeProvider,
    weak: true,
    (_, next) {
    final value = next.valueOrNull?.firstWhereOrNull((item) => item.id == id);
    if (value != null) ref.state = value;
  });

}

Of course, this is fairly low-level. I'm thinking about adding a higher-level API around this to make it simpler.

In particular, I'm considering adding a "Repository" automatically generated by riverpod_generator, which would handle a full CRUD for an entity, along with paginated APIs.

[mattermoran]

Having generated repository for entities sounds fantastic and most likely cover all those cases

[lucavenir]

Sorry bout the late relpy! For @mattermoran:

As to how exactly to do this is up to you.

Well ofc. My point is: It seems I can't find an idiomatic way to do so.

In my case I use graphql, hook into response coming from api, and then place it into my cache that is then read by a provider.

See? That's the point. Riverpod is A Reactive Caching and Data-binding Framework. So why must we re-implement our own cache, again? (I suppose via a database / persistent storage of some kind).

Anyways, your following comment brings up a suggestion for @rrousselGit:

This is a very basic explanation as I do bunch of other stuff like normalizing data to handle deeply nested objects, merging data so incompletely data doesn't just override, etc.
This comes with bunch of other advantages like navigating around the app with just ids as you always know where to get that data from, "free" offline mode as you already have the data cached, etc

This should be addressed in the documentation: if Riverpod is meant to be used with some data normalization, or any best practice really, it should be explicitly told in a dedicated documentation page.

[lucavenir]

And about riverpod 3's new API @rrousselGit just showed: again, I'm not sure how this solves the above problem.

The main pain point of this issue is: even after data normalization, I am unable to determine which provider should be watched. Here you're suggesting something like:

// weak listen is great, but we're still referencing one single list
watched.valueOrNull?.firstWhereOrNull((item) => item.id == id); 

This proposal is cool, but because of points (1) and (2) of my message above, but I can't see how it addressed the "synced state problem".

Because of data normalization, I would love to do the following instead:

@Riverpod(lazy: true)
int favoriteId(FavoriteIdRef ref, int id) {
  return Uninitialized<int>;  // sentinel value, or something similar
}

// somewhere else
@riverpod
List<int> someList(SomeListRef ref) async {
  final elements = await fetch();
  for (final element in elements) {
    ref.listen(favoriteId(id).initializeWith(element.favoriteId), (previous, next) {...});
  }
}

But I'm not sure this is even doable. Maybe with metaprogramming we could define uninitialized / lazy providers via typedefs annotations, somehow:

// defining a typedef instead of a function means defining a provider with no initialization (aka lazy)
@riverpod
typedef FavoriteIdProvider = int Function(Ref ref, int id);

And let static metaprogramming do its thing. But again, I'm being imaginative here.
Honestly, any working suggestion is welcome. Besides implementing my own second cache on my own, that is 😄

[rrousselGit]

With normalized data, you have only a single source of truth for what an entity is.

There's no case of "Item in home vs Item in detail page". They all should be using the exact same instance.
There shouldn't be a case where mutating one Item(id: 42) isn't reflected on other Item(id: 42), since there's only a single instance of it.

[lucavenir]

There shouldn't be a case where mutating one Item(id: 42) isn't reflected on other Item(id: 42), since there's only a single instance of it.

Yes this is very clear. This is what we're aiming at. This is why we're defining this ItemController with the "change favorite status mutation" on it.
The question in this thread is: how? (i.e. with riverpod)
The .firstWhereOrNull solution might bring to inconsistent state - if I've understood this correctly - so it's not really viable. Also, which list and which page should I watch?.

They all should be using the exact same instance
There's no case of "Item in home vs Item in detail page"

I think I might have a know-how gap I might need to fill.
Please, I'd be thankful if you'd help me out with this / point me towards some readings afterwards.
But.

Here's my POV, with a pragmatic problem to solve:

These two lists do come from semantically different sources, tho. Say I set to favorite my own item.
Such item now belongs to faovritesProvider(page: 5), while being on myItemsProvider(page: 2), but also casually appearing on homeProvider(page: 0).

The only way - that I currently see - is to have "one single source of truth" is to have an ItemController, but - again - the problem is its initialization..

[lucavenir]

Hi 😸

I'm back on this issue, I re-read the whole conversations. Me and some colleagues tried some solutions towards this "synced-state-problem", but nothing is really working out.

I'm just curious - is this really a "riverpod" problem? Or is it a more generalized problem?
Because I can see our issue reproduced on several social platforms, e.g. Twitter's like system.

So I'm wondering if riverpod is ever considering to find a solution for this problem.
Or if it's relevant to riverpod at all.
Answering this would greatly help our R&D process. We might change everything just to tackle this one out.
And, consequently, we might close this issue with a "wont-fix" label.

[rrousselGit]

The problem is by no means unique to Riverpod. It's a general software issue

That doesn't mean Riverpod won't try to solve this. That's what Riverpod does anyway, trying to solve general software issues. That's why Riverpod also will add things like offline persistence & stuff. Not unique to Riverpod, but good to have

[lucavenir]

Thank you for replying so fast!

I'm glad to know the issue is acknowledged then 😃

[lucavenir]

Hello there! First of all thank you for solving this.
While we wait for Riverpod 3: I still can't fully understand how to solve this, given I can use the new weak: true API.

It's been a while, so here's the TLDR of the problem, hoping that you could clarify this.

An app shows a list of Books in its home page, but also allows me to search for a Book in a search page. Then again, each book has its detail page. Finally, there's a Library page, showing different libraries, each with their list view of Books. Every list mentioned above is paginated.

Here's the twist: every book is likeable, aka all entities here share this common property.
How can I properly keep track of a book's bool isFavorite property, in every different page?
A typical user flow is, e.g., a book is liked in the library page; pressing "back", the home shows this book's favorite state updated accordingly.

Here's an additional twist, regarding our source of truth: While the user navigates the app, state changes might occur in other places as the navigation occurs (e.g. in a separate web app). So to optimize for our user's experience we need to implement a "WYSIWYG" approach, in which, the latest state received from our server is "the truth".

How can I solve this problem using listen: weak?

[w0rsti]

As we are basically talking about related issues (#3781) I am currently facing the same issue.

I think how a weak listener can help is as follows (sorry I am on the phone so no code examples):

All mutations on an item should happen in the detailed provider/notifier. E.g. your update function on to like an item should be within the ItemNotifier.

This way all other providers can weakly listen to the according provider and update their internal state to match it to the ItemNotifier one.

So your PaginatedProvider fetched a List which then listens weakly to every ItemNotifier and if this notifier exposes a new state of an item in its list, it will update its state (the list of item) with the updated item.