Feature/async iterator (#29)

* Allow use of for await ... of operator on ODataQuery

* fix paging with async iterable

* Fix rebase issues

* Update documentation for new async iterator feature

README.md

@@ -6,17 +6,21 @@
 A library for creating and executing OData queries. It makes heavy use of TypeScript for a better developer experience.
 
 ## Installation
+
 ```console
 npm install ts-odata-client
 ```
 
 ## Prerequisites
-This library does not have any dependencies on any other NPM packages, but it does utilize the `fetch` and `Proxy` APIs. A pollyfil can be used for `fetch`, but there is currently no known pollyfil for `Proxy`; see [browser support](https://caniuse.com/?search=Proxy()).
+
+This library does not have any dependencies on any other NPM packages, but it does utilize the `fetch` and `Proxy` APIs. A pollyfil can be used for `fetch`, but there is currently no known pollyfil for `Proxy`; see [browser support](<https://caniuse.com/?search=Proxy()>).
 
 ## Supported Versions
+
 Only OData version 4 is currently supported by this library.
 
 ## Quick Start
+
 If you want to start coding immediately, install the NPM package, then use the following as reference.
 
 ```ts
@@ -28,12 +32,41 @@ const results = await ODataQuery.forV4<User>('http://domain.example/path/to/endp
   .filter(u => u.firstName.$startsWith('St').and(u.age.$greaterThanOrEqualTo(25))
   .select(u => ( { givenName: u.firstName, surname: u.lastName } ))
   .getManyAsync();
-  
+
   console.log(results); // results is the json object that is returned by the OData service
   console.log(results.value); // results.value is of type Array<{givenName: string, surname: string}>
 ```
 
+## Async Iterable
+
+The `ODataQuery` class is an async iterable object. This means you can start iterating over it with an async for loop.
+
+```ts
+const usersQuery = ODataQuery.forV4<User>(...);
+
+for await (const user of usersQuery) {
+  // This will iterate over all of the users returned from the server.
+  // If paging is enabled on the server, then the next page will only be fetched when it is needed.
+}
+```
+
+## Server Paging
+
+If paging is enabled on the server, the result of calling `getManyAsync` and `getManyWithCountAsync` on `ODataQuery` will have a `next()` function defined. Invoking that will fetch the next page of data from the server.
+
+```ts
+const usersQuery = ODataQuery.forV4<User>(...);
+
+const queryResult = await usersQuery.getManyAsync();
+// queryResult.value will contain the results returned from the server
+if(queryResult.next !== undefined) {
+  const page2Result = await queryResult.next();
+  // page2Result.value will contain the results on the second page from the server
+}
+```
+
 ## Using a Data Context
+
 The above is great for one-off queries, but if you have an OData service with mutliple endpoints, and you want to encapsulate that in a single class, then write a class that extends the provided `ODataContextV4` class
 
 ```ts
@@ -45,7 +78,7 @@ class MyODataContext extends ODataV4Context {
   constructor(baseUrl: string) {
     super(baseUrl/*, options object if needed*/);
   }
-  
+
   get users() { return this.createQuery<User>('relative/path/from/baseUrl'); }
 }
 
@@ -57,70 +90,76 @@ const result = await context.users
 ```
 
 ## Executing OData Functions
+
 An OData function that returns a collection from an entity set can be treated just like any other endpoint, but it has the advantage that it can take parameter values
 
 ```ts
-function callODataFunction(parameter: string)
+function callODataFunction(parameter: string);
 // Remember: If the parameter is a string and it has a single quote in it, that will need to be escaped with two single quotes
-const query = ODataQuery.forV4<User>(`http://domain.example/path/to/endpoint/function(myParameter='${parameter}')`/*, options object if needed*/);
+const query = ODataQuery.forV4<User>(
+  `http://domain.example/path/to/endpoint/function(myParameter='${parameter}')` /*, options object if needed*/,
+);
 ```
 
 `query` can now be used like any other OData Query (e.g., `filter`, `select`, `top`, etc.).
 
-
 ## Query Builder
 
 This library also provides a query builder option that returns only the plain filter and query object, ensuring type safety. Utilizing the same syntax as `ODataQuery`, you can create precise filter expressions and query objects, For example:
 
 ```ts
 // Create a type-safe filter expression
 const result = ODataExpression.forV4<User>()
-    .filter((p) => p.firstName.$equals("john"))
-    .build();
+  .filter((p) => p.firstName.$equals("john"))
+  .build();
 
-console.log(results); 
+console.log(results);
 // Output: { "$filter": "firstName eq 'john'", ... }
 ```
 
 By employing the query builder, you can adhere to the familiar syntax of `ODataQuery` while obtaining a streamlined result containing only the essential filter and query information.
 
 ## Upgrading from v1.x to 2.x
+
 2.0 introduces a number of breaking changes. The primary breaking changes are with the `filter`, `orderBy` and `orderByDescending` methods on the `ODataQuery` type.
 
 ### filter
-  This method still accepts a `BooleanPredicateBuilder<T>` as an argument; however, the method signature alternative has now changed; instead of a `FilterBuilder` object, the provided argument for the method is now an `EntityProxy`. The best way to demonstrate the difference is with an example.
-  
-  ```ts
-  const userQuery = ...
-
-  //v1.x syntax:
-  userQuery.filter(f => f.greaterThan('firstName', 'St').and(f.equals('age', 30)));
-
-  //v2.x syntax:
-  userQuery = filter(u => u.firstName.$greaterThan('St').and(u.age.$equals(30)));
-
-  //v2.x alternative conjunction syntax
-  userQuery.filter((u, {and}) => and(u.firstName.$greaterThan('St'), u.age.$equals(30));  
-  ```
-  
-  Note: in the last example, the second parameter `{and}` is a destructuring of the `FilterAccessoryFunctions` type, currently supported methods are `and`, `or`, and `not`. While `and` and `or` can be handled without the second argument (e.g., `userQuery = filter(u => u.firstName.$greaterThan('St').and(u.age.$equals(30)));`), the `not` method is only available from the `FilterAccessoryFunctions` type.
-  
+
+This method still accepts a `BooleanPredicateBuilder<T>` as an argument; however, the method signature alternative has now changed; instead of a `FilterBuilder` object, the provided argument for the method is now an `EntityProxy`. The best way to demonstrate the difference is with an example.
+
+```ts
+const userQuery = ...
+
+//v1.x syntax:
+userQuery.filter(f => f.greaterThan('firstName', 'St').and(f.equals('age', 30)));
+
+//v2.x syntax:
+userQuery = filter(u => u.firstName.$greaterThan('St').and(u.age.$equals(30)));
+
+//v2.x alternative conjunction syntax
+userQuery.filter((u, {and}) => and(u.firstName.$greaterThan('St'), u.age.$equals(30));
+```
+
+Note: in the last example, the second parameter `{and}` is a destructuring of the `FilterAccessoryFunctions` type, currently supported methods are `and`, `or`, and `not`. While `and` and `or` can be handled without the second argument (e.g., `userQuery = filter(u => u.firstName.$greaterThan('St').and(u.age.$equals(30)));`), the `not` method is only available from the `FilterAccessoryFunctions` type.
+
 ### orderBy and orderByDescending
-  Similar to filter, these methods now take an `EntityProxy` type as the method parameter.
-  
-  ```ts
-  const userQuery = ...
-
-  //v1.x syntax:
-  userQuery.orderBy('firstName');
-
-  //v2.x syntax:
-  userQuery.orderBy(u => u.firstName);
-  //to sort on multiple properties
-  userQuery.orderBy(u => [u.lastName, u.firstName]);
-  ```
+
+Similar to filter, these methods now take an `EntityProxy` type as the method parameter.
+
+```ts
+const userQuery = ...
+
+//v1.x syntax:
+userQuery.orderBy('firstName');
+
+//v2.x syntax:
+userQuery.orderBy(u => u.firstName);
+//to sort on multiple properties
+userQuery.orderBy(u => [u.lastName, u.firstName]);
+```
 
 ### New select overload
+
 The `select` method maitains backwards compatibility, so no change is needed to existing code when updating, but an overload has been added that is more powerful than the one in version 1.x.
 
 ```ts
@@ -142,7 +181,9 @@ console.log(result); //{result: [{managerLastName: string}, ...]}
 The v1.x syntax only allows you to pick and choose which top-level entity properties are returned. The v2.x syntax allows you to choose nested properties AND allows you to change the shape of what is returned to your code after it executes the query.
 
 #### Important Notes/Limitations
+
 Please note the following when using the newer style syntax:
+
 1. JavaScript/TypeScript does not support a true expression syntax that allows the content of the method you provide to be inspected. For best results, simply return an object literal from the method and avoid attempting to do anything with the entity values other than assigning them directly to a field or an array.
 1. Note the `()` surrounding the `{}` in the arrow method body. This is needed; wthout it, JavaScript/TypeScript assumes the `{}` are defining a new block, NOT an object literal.
 1. The OData request will include the correct `$select` parameter, only returning the data that is needed for your custom object.

src/lib/ODataQuery.ts

@@ -12,7 +12,7 @@ import { resolveQuery, type ReplaceDateWithString } from "./ProxyFilterTypes";
  * Represents a query against an OData source.
  * This query is agnostic of the version of OData supported by the server (the provided @type {ODataQueryProvider} is responsible for translating the query into the correct syntax for the desired OData version supported by the endpoint).
  */
-export class ODataQuery<T, U = ExcludeProperties<T, unknown[]>> extends ODataQueryBase<T, U> {
+export class ODataQuery<T, U = ExcludeProperties<T, []>> extends ODataQueryBase<T, U> {
   static forV4<T>(endpoint: string, options?: Partial<ODataV4Options>) {
     return new ODataQuery<T>(new ODataV4QueryProvider(endpoint, options));
   }
@@ -30,7 +30,6 @@ export class ODataQuery<T, U = ExcludeProperties<T, unknown[]>> extends ODataQue
    */
   public async getAsync(key: unknown) {
     const expression = new Expression(ExpressionOperator.GetByKey, [key], this.expression);
-    // return await this.provider.executeQueryAsync<ODataResponse & ReplaceDateWithString<U>>(expression);
     const result = await this.provider.executeQueryAsync<ODataResponse & ReplaceDateWithString<U>>(expression);
     const selectMap = getSelectMap(expression);
     if (selectMap == null) return result;
@@ -47,10 +46,7 @@ export class ODataQuery<T, U = ExcludeProperties<T, unknown[]>> extends ODataQue
     const results = await this.provider.executeQueryAsync<ODataQueryResponse<ReplaceDateWithString<U>>>(
       this.expression,
     );
-    const selectMap = getSelectMap(this.expression);
-    if (selectMap != null) {
-      results.value = results.value.map(selectMap) as unknown as ReplaceDateWithString<U>[];
-    }
+    handleODataQueryResults(this.provider, this.expression, results);
     return results;
   }
 
@@ -61,10 +57,8 @@ export class ODataQuery<T, U = ExcludeProperties<T, unknown[]>> extends ODataQue
     const expression = new Expression(ExpressionOperator.GetWithCount, [], this.expression);
     const results =
       await this.provider.executeQueryAsync<ODataQueryResponseWithCount<ReplaceDateWithString<U>>>(expression);
-    const selectMap = getSelectMap(expression);
-    if (selectMap != null) {
-      results.value = results.value.map(selectMap) as unknown as ReplaceDateWithString<U>[];
-    }
+    handleODataQueryResults(this.provider, this.expression, results);
+
     return results;
   }
 
@@ -76,6 +70,20 @@ export class ODataQuery<T, U = ExcludeProperties<T, unknown[]>> extends ODataQue
   [resolveQuery]() {
     return this.provider.buildQuery(this.expression);
   }
+
+  // enables usage of for await ... of operator
+  [Symbol.asyncIterator]() {
+    return odataAsyncIterator(this);
+  }
+}
+
+async function* odataAsyncIterator<T, U>(query: ODataQuery<T, U>) {
+  let results = await query.getManyAsync();
+  do {
+    yield* results.value;
+    if (results.next == null) return undefined;
+    results = await results.next();
+  } while (true);
 }
 
 function getSelectMap<T, U>(expression?: Expression): ((entity: T) => U) | undefined {
@@ -88,3 +96,27 @@ function getSelectMap<T, U>(expression?: Expression): ((entity: T) => U) | undef
   }
   return;
 }
+
+function handleODataQueryResults<T>(
+  provider: ODataQueryProvider,
+  expression: Expression | undefined,
+  result: ODataQueryResponse<T>,
+) {
+  if (expression != null) applySelectMapIfExists(expression, result);
+  if (result["@odata.nextLink"] != null) {
+    result.next = async () => {
+      const nextPageResult = (await provider.executeQueryAsync(
+        result["@odata.nextLink"] as string,
+      )) as ODataQueryResponse<T>;
+      handleODataQueryResults(provider, expression, nextPageResult);
+      return nextPageResult;
+    };
+  }
+}
+
+function applySelectMapIfExists<T>(expression: Expression, results: ODataQueryResponse<T>) {
+  const mapper = getSelectMap<T, T>(expression);
+  if (mapper == null) return results;
+  results.value = results.value.map(mapper);
+  return results;
+}

src/lib/ODataQueryBase.ts

@@ -29,7 +29,7 @@ export class ODataQueryBase<T, U = ExcludeProperties<T, unknown[]>> {
   public select<U extends FieldsFor<T>>(...fields: U[]): ODataQueryBase<T, U>;
   public select<U extends ProjectorType>(projector: (proxy: T) => U): ODataQueryBase<T, U>;
   public select<U>(...args: [(proxy: T) => U | FieldsFor<T>, ...FieldsFor<T>[]]) {
-    if (args.length === 0) throw new Error("Parameters are requird");
+    if (args.length === 0) throw new Error("Parameters are required");
 
     const firstArg = args[0];
     if (typeof firstArg === "function") {

src/lib/ODataQueryProvider.ts

@@ -25,6 +25,12 @@ export abstract class ODataQueryProvider {
    */
   abstract executeQueryAsync<T extends ODataResponse>(expression?: Expression): Promise<T>;
 
+  /**
+   * Fetches the OData response from the provided URL. Primarily used to fetch subsequent pages
+   * @param url
+   */
+  abstract executeQueryAsync<T extends ODataResponse>(odataUrl: string): Promise<T>;
+
   /**
    * Executed the provided @type {Expression} and returns the raw response.
    * @param expression

src/lib/ODataResponse.ts

@@ -17,6 +17,11 @@ export interface ODataQueryResponse<T> extends ODataResponse {
    */
   ["@odata.nextLink"]?: string;
 
+  /**
+   * If server-side paging is implemented, and the server returned a value for "@odata.nextLink", this method will fetch the next paging of data using that link.
+   */
+  next?(): Promise<ODataQueryResponse<T>>;
+
   /**
    * The results of the OData query
    */

src/lib/ODataV4QueryProvider.ts

@@ -22,28 +22,32 @@ export class ODataV4QueryProvider extends ODataQueryProvider {
   }
 
   static createQuery<T>(path: string, options?: Partial<ODataV4Options>) {
-    return new ODataV4QueryProvider(path, options).createQuery<T, ExcludeProperties<T, unknown[]>>();
+    return new ODataV4QueryProvider(path, options).createQuery<T, ExcludeProperties<T, []>>();
   }
 
-  private async sendRequest(expression?: Expression) {
-    const url = this.buildQuery(expression);
-
+  private async sendRequest(url: string) {
     let init = this.options?.requestInit?.() ?? {};
     if (init instanceof Promise) init = await init;
 
     return await fetch(url, init);
   }
 
-  async executeQueryAsync<T extends ODataResponse>(expression?: Expression) {
-    const response = await this.sendRequest(expression);
+  executeQueryAsync<T extends ODataResponse>(expression?: Expression): Promise<T>;
+  executeQueryAsync<T extends ODataResponse>(odataUrl: string): Promise<T>;
+  async executeQueryAsync<T extends ODataResponse>(value?: Expression | string): Promise<T> {
+    if (typeof value !== "string") value = this.buildQuery(value);
+    const response = await this.sendRequest(value);
 
-    if (response.ok) return (await response.json()) as T;
+    if (response.ok) {
+      return (await response.json()) as T;
+    }
 
     throw new Error(JSON.stringify(await response.json()));
   }
 
   async executeRequestAsync(expression?: Expression) {
-    const response = await this.sendRequest(expression);
+    const url = this.buildQuery(expression);
+    const response = await this.sendRequest(url);
 
     if (response.ok) return response;
 
@@ -86,7 +90,8 @@ export class ODataV4QueryProvider extends ODataQueryProvider {
   }
 
   private buildQueryString(query: ODataV4QuerySegments) {
-    const queryString: string[] = [];
+    // Not using URLSearchParams as it will escape the leading $ character in the querystring name, which is not desired
+    const queryString = new Array<string>();
 
     if (query.filter) queryString.push(`$filter=${encodeURIComponent(query.filter)}`);