Skip to content

Commit

Permalink
Updated documents per comments
Browse files Browse the repository at this point in the history
  • Loading branch information
XiaoningLiu authored and vinjiang committed May 9, 2019
1 parent a1d2695 commit 6e64127
Show file tree
Hide file tree
Showing 4 changed files with 80 additions and 65 deletions.
3 changes: 3 additions & 0 deletions BreakingChanges.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Breaking Changes

> Note. This file includes breaking changes after 3.0.0-preview. For legacy Azurite changes, please goto GitHub [releases](https://github.com/Azure/Azurite/releases).
2 changes: 2 additions & 0 deletions CONTRIBUTION.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Contribution

> Go to [GitHub project](https://github.com/Azure/Azurite/projects) page or [GitHub issues](https://github.com/Azure/Azurite/issues) for the milestone and TODO items we are used for tracking upcoming features and bug fixes.
## Development Environments

Visual Studio Code is the best editor for TypeScript. Install and configure following plugins and configurations before starting development.
Expand Down
7 changes: 7 additions & 0 deletions ChangeLog.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Changelog

> Note. This file includes changes after 3.0.0-preview. For legacy Azurite changes, please goto GitHub [releases](https://github.com/Azure/Azurite/releases).
2019.05 Version 3.0.0-preview

- Initial Release of Azurite V3.
133 changes: 68 additions & 65 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,18 +5,18 @@
> Master branch has been updated with latest Azurite V3.
> V3 currently only supports Blob service, please use V2 for Queue or Table service for the time being.
| Version | Azure Storage API Version | Service Support | Description | Reference Links |
| ------------- | ------------------------- | -------------------- | ------------------------------------------------- | -------------------------------------------- |
| 3.0.0-preview | 2018-03-28 | Blob | Azurite V3 based on TypeScript & New Architecture | [NPM](https://www.npmjs.com/package/azurite) |
| 2.7.0 | 2016-05-31 | Blob, Queue and File | Legacy Azurite V2 | [NPM](https://www.npmjs.com/package/azurite) |
| Version | Azure Storage API Version | Service Support | Description | Reference Links |
| ------------- | ------------------------- | --------------------- | ------------------------------------------------- | -------------------------------------------- |
| 3.0.0-preview | 2018-03-28 | Blob | Azurite V3 based on TypeScript & New Architecture | [NPM](https://www.npmjs.com/package/azurite) |
| 2.7.0 | 2016-05-31 | Blob, Queue and Table | Legacy Azurite V2 | [NPM](https://www.npmjs.com/package/azurite) |

## Introduction

Azurite is an open source Azure Storage API compatible server (emulator). Based on Node.js, Azurite provides cross platform experiences for customers wanting to try Azure Storage easily in a local environment. Azurite simulates most of the commands supported by Azure Storage with minimal dependencies.

Azurite V2 is manually created with pure JavaScript, popular and active as an open source project. However, Azure Storage APIs are growing and keeping updating, manually keeping Azurite up to date is not efficient and prone to bugs. JavaScript also lacks strong type validation which prevents easy collaboration.

Compared to V2, Azurite V3 implements a new architecture leveraging code generated by a TypeScript Server Code Generator we created. The generator uses the same swagger (modified) used by the new Azure Storage SDKs. This reduces manual effort and more facilitates better code alignment with storage APIs.
Compared to V2, Azurite V3 implements a new architecture leveraging code generated by a TypeScript Server Code Generator we created. The generator uses the same swagger (modified) used by the new Azure Storage SDKs. This reduces manual effort and more facilitates better code alignment with storage APIs.

3.0.0-preview is the first release version using Azurite's new architecture.

Expand All @@ -31,7 +31,7 @@ Compared to V2, Azurite V3 implements a new architecture leveraging code generat
- Features **NEW** on V3
- Built with TypeScript and ECMA native promise and async features
- New architecture based on TypeScript server generator. Leverage auto generated protocol layer, models, serializer, deserializer and handler interfaces from REST API swagger
- Flexible structure and architecture, supports customizing handler layer implementation, persistency layer implementation, multi storage accounts, HTTP pipeline middleware injection
- Flexible structure and architecture, supports customizing handler layer implementation, persistency layer implementation, HTTP pipeline middleware injection
- Detailed debugging log support, easy bug locating and reporting
- Works with storage .Net SDK basic and advanced sample
- SharedKey, AccountSAS, ServiceSAS, Public Access authentication support
Expand All @@ -43,11 +43,13 @@ Try with any of following ways to start an Azurite V3 instance.

### GitHub

After cloning source code, execute following commands to start Azurite V3. Take blob service as example:
After cloning source code, execute following commands to install and start Azurite V3.

```bash
npm install
npm run blob
npm run build
npm install -g
azurite
```

### NPM
Expand All @@ -74,54 +76,29 @@ azurite --silent --location c:\azurite --debug c:\azurite\debug.log

This tells Azurite to store all data in a particular directory `c:\azurite`. If the `-l` option is omitted it will use the current working directory. You can also selectively start different storage services.

For Blob Storage Service only:
For example, to start blob service only:

```bash
$ azurite-blob -l path/to/azurite/workspace
```

### DockerHub

_Will release Azurite V3 to DockerHub soon._
> Will support more release channels for Azurite V3 in the future.
### Visual Studio Code Extension
### DockerHub

_Plan to release Azurite V3 as a Visual Studio Code Extension soon._
_Releasing Azurite V3 to DockerHub is under investigation._

### NuGet

_Investigating options to release Azurite V3 to NuGet._

## Usage with Azure Storage SDKs or Tools

### Default Storage Account

Azurite V3 provides support for a default storage account as General Storage Account V2 and associated features.

- Account name: `devstoreaccount1`
- Account key: `Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==`

> Note. Besides SharedKey authentication, Azurite V3 supports account and service SAS authentication. Anonymous access is also available when container is set to allow public access.
### Connection String

Typically you can pass following connection strings to SDKs or tools (like Azure CLI2.0 or Storage Explorer)
_Releasing Azurite V3 to NuGet is under investigation._

Take blob service as example, full connection string is:

```
DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://127.0.0.1:10000/devstoreaccount1;
```

Or if the SDK or tools support following short connection string:
### Visual Studio Code Extension

```
UseDevelopmentStorage=true;
```
_Publish Azurite as a Visual Studio Code Extension is under investigation._

### Storage Explorer
### Visual Studio

Connect to Azurite by click "Add Account" icon, then select "Attach to a local emulator" and click "Connect".
_Integrate Azurite with Visual Studio is under investigation._

## Supported Command Line Options

Expand All @@ -147,6 +124,8 @@ You can customize the listening address per your requirements.
Optional. By default, Azurite V3 will listen to 10000 as blob service port.
You can customize the listening port per your requirements.

> Warning: After using a customized port, you need to update connection string or configurations correspondingly in your Storage Tools or SDKs.
#### Customize Blob Service Listening Port

```cmd
Expand All @@ -158,6 +137,7 @@ You can customize the listening port per your requirements.
```cmd
--blobPort 0
```

Note: The port in use is displayed on Azurite startup.

### Workspace Path Configuration
Expand Down Expand Up @@ -198,15 +178,46 @@ Azurite V3 supports SharedKey, Account Shared Access Signature (SAS), Service SA

An option to bypass authentication is **NOT** provided in Azurite V3.

## Differences between Azurite and Azure Storage Service
## Usage with Azure Storage SDKs or Tools

### Default Storage Account

Azurite V3 provides support for a default storage account as General Storage Account V2 and associated features.

- Account name: `devstoreaccount1`
- Account key: `Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==`

> Note. Besides SharedKey authentication, Azurite V3 supports account and service SAS authentication. Anonymous access is also available when container is set to allow public access.
### Connection String

Typically you can pass following connection strings to SDKs or tools (like Azure CLI2.0 or Storage Explorer)

Take blob service as example, full connection string is:

```
DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://127.0.0.1:10000/devstoreaccount1;
```

Or if the SDK or tools support following short connection string:

```
UseDevelopmentStorage=true;
```

### Storage Explorer

Connect to Azurite by click "Add Account" icon, then select "Attach to a local emulator" and click "Connect".

## Differences between Azurite and Azure Storage

Because Azurite runs as a local instance for persistent data storage, there are differences in functionality between Azurite and an Azure storage account in the cloud.

### Storage Accounts

> Please reach to us or open issues if you need multi storage account support.
Azurite V3 supports a default account as General Storage Account V2 and provide features.
Azurite V3 supports a default account as General Storage Account V2 and provides features.

- Account name: `devstoreaccount1`
- Account key: `Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==`
Expand Down Expand Up @@ -239,7 +250,7 @@ For example, the following address might be used for accessing a blob in Azurite
http://127.0.0.1:10000/myaccount/mycontainer/myblob.txt
```

The service endpoints for Azurite Blob service:
The service endpoints for Azurite blob service:

```
http://127.0.0.1:10000/<account-name>/<resource-path>
Expand All @@ -255,28 +266,18 @@ Azurite is not a scalable storage service and does not support many concurrent c

> Please reach to us if you have requirements or suggestions for a specific error handling.
Azurite trys to align with Azure Storage error handling logic, but CANNOT provide 100% alignment, such as error messages (returned in error response body) maybe different (while error status code will align). Because of following reasons:

- No clear rules defined on Azure Storage online documentation
- Many logical errors cannot be defined in swagger definitions
- For a request with more than 1 logical error, it’s up to implementation to decide which error is thrown first
- Other structure related unexpected errors proposed by Azure Storage, like under layer errors

There is a typical example describing the challenge to fully align with server behavior. For download blob range operation, when the request range headers exceed the blob length. Azure Storage server will just return the data it has, instead returning an error. While this behavior doesn’t get described in the REST documents.
Azurite tries to align with Azure Storage error handling logic, and provides best-efforts alignment based on Azure Storage online documentation. But CANNOT provide 100% alignment, such as error messages (returned in error response body) maybe different (while error status code will align).

### API Version Compatible Strategy

Azurite V3 follows a **Try best to serve** compatible strategy with Azure Storage API versions:

- An Azurite V3 instance has a baseline Azure Storage API version.
- An Azurite V3 instance has a baseline Azure Storage API version.
- A Swagger definition (OpenAPI doc) with the same API version will be used to generate protocol layer APIs and interfaces.
- Azurite should implement all the possible features provided in this API service version.
- If an incoming request has **the same API version** Azurite provides, Azurite should handle the request with parity to Azure Storage
- If an incoming request has **the same API version** Azurite provides, Azurite should handle the request with parity to Azure Storage.
- If an incoming request has a **higher API version** than Azurite, Azurite will return a VersionNotSupportedByEmulator error (HTTP status code 400 - Bad Request).
- If an incoming request has a **lower API version** header than Azurite, Azurite will attempt to handle the request with Azurite’s baseline API version behavior instead of that specified in the request.
- If the REST API keeps the same behavior in these 2 API versions, Azurite should handle the request well. This covers most of the REST APIs.
- If the REST API behaviors different between 2 versions (breaking changes), Azurite will not behave in the same way as the old API version.
- Minor APIs have breaking changes cross API versions. Possibly affected APIs are those dealing with ETag
- Azurite will return API version in response header as the baseline API version
- SAS accepts pattern from API version 2015-04-05

Expand All @@ -298,7 +299,7 @@ Architecture in Azurite V3 has been refactored, it's more flexible and robust. I

- Use other HTTP frameworks instead of express.js
- Customized new handler layer implementation, such as redirecting requests to Azure Storage services
- Implement and inject a new persistency layer implementation, such as one based on a different database service
- Implement and inject a new persistency layer implementation, such as one based on a different database service
- Provide support for multiple azure storage accounts and authentication
- Detailed debug logging for easy issue investigation and request tracking
- Create HTTPS server
Expand All @@ -315,22 +316,22 @@ Azurite V3 selected TypeScript as its' programming language, as this facilitates
### Features Scope

Legacy Azurite V2 supports Azure Storage Blob, Queue and Table services.
Azurite V3 currently only supports Azure Storage Blob Service, with queue support to follow soon.
Table service support is currently under discussion.
Azurite V3 currently only supports Azure Storage blob service, with queue support to follow soon.
Table service support is currently under discussion.

Azurite V3 supports features from Azure Storage API version 2018-03-28, and willmaintain parity with the latest API versions, in a more frequent update frequency than legacy Azurite V2.
Azurite V3 supports features from Azure Storage API version 2018-03-28, and will maintain parity with the latest API versions, in a more frequent update frequency than legacy Azurite V2.

## TypeScript Server Code Generator

Azurite V3 leverages a TypeScript Node.js Server Code Generator to generate the majority of code from Azure Storage REST APIs swagger specification.
Currently, the generator project is private, under development and only used by Azurite V3.
Currently, the generator project is private, under development and only used by Azurite V3.
We have plans to make the TypeScript server generator public after Azurite V3 releases.
All the generated code is kept in `generated` folder, including the generated middleware, request and response models.

## Support Matrix

3.0.0-preview release targets **2018-03-28** API version **blob** service.
Detailed support matrix:
Detailed support matrix:

- Supported Vertical Features
- SharedKey Authentication
Expand Down Expand Up @@ -378,9 +379,11 @@ This project is licensed under MIT.

## We Welcome Contributions!

> Go to [GitHub project](https://github.com/Azure/Azurite/projects) page or [GitHub issues](https://github.com/Azure/Azurite/issues) for the milestone and TODO items we are used for tracking upcoming features and bug fixes.
We are currently working on Azurite V3 to implement the remaining Azure Storage REST APIs.
We finished the basic structure and majority of features in Blob Storage, as can be seen in the support matrix.
The detailed work items are also tracked in GitHub repository projects and issues.
The detailed work items are also tracked in GitHub repository projects and issues.

Any contribution and suggestions for Azurite V3 is welcome, please goto `CONTRIBUTION.md` for detailed contribution guidelines. Alternatively, you can open GitHub issues voting for any missing features in Azurite V3.

Expand Down

0 comments on commit 6e64127

Please sign in to comment.