Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Blob upload failing on LTS_08_2024 #2658

Open
shekharg987 opened this issue Nov 20, 2024 · 1 comment
Open

Blob upload failing on LTS_08_2024 #2658

shekharg987 opened this issue Nov 20, 2024 · 1 comment
Assignees
@ewertons
Labels
question

Comments

@shekharg987
Copy link

shekharg987 commented Nov 20, 2024 •
edited
Loading

Using ubuntu 18.0 version upgrading SDK to latest LTS release LTS_03_2024

Using X509 based authentication facing blob upload failing from SDK side

Below ae the console logs

[ERROR] Socketio_Failure: Receiving data from endpoint: errno=104.
[ERROR] Error signalled by underlying IO
[ERROR] xio reported error on dowork
[ERROR] Receive header from HTTP failed (result = HTTPAPI_READ_DATA_FAILED (10))
[ERROR] unable to recover sending to a working state
[ERROR] unable to HTTPAPIEX_ExecuteRequest
[ERROR] unable to HTTPAPIEX_ExecuteRequest
[ERROR] error in IoTHubClient_LL_UploadToBlob_GetBlobCredentialsFromIoTHub
[ERROR] Failed initializing upload in IoT Hub
[ERROR] unable to IoTHubClientCore_LL_UploadToBlob
[ERROR] fail to upload blob

SDK side logs
ERROR [dk-c\c-utility\adapters\socketio_win32.c:700 ] Socketio_Failure: Receiving data from endpoint: 10054.
ERROR [k-c\c-utility\adapters\httpapi_compact.c:476 ] Error signalled by underlying IO
ERROR [dk-c\c-utility\adapters\socketio_win32.c:700 ] Socketio_Failure: Receiving data from endpoint: 10054.
[k-c\c-utility\adapters\httpapi_compact.c:566 ] xio reported error on dowork
ERROR [k-c\c-utility\adapters\httpapi_compact.c:1276] Receive header from HTTP failed (result = HTTPAPI_READ_DATA_FAILED (10))
ERROR [zure-iot-sdk-c\c-utility\src\httpapiex.c:551 ] unable to recover sending to a working state
ERROR [ient\src\iothub_client_ll_uploadtoblob.c:156 ] unable to HTTPAPIEX_ExecuteRequest
ERROR [ient\src\iothub_client_ll_uploadtoblob.c:546 ] unable to HTTPAPIEX_ExecuteRequest
ERROR [ient\src\iothub_client_ll_uploadtoblob.c:915 ] error in IoTHubClient_LL_UploadToBlob_GetBlobCredentialsFromIoTHub
ERROR [othub_client\src\iothub_client_core_ll.c:2728] Failed initializing upload in IoT Hub
ERROR [c\iothub_client\src\iothub_client_core.c:2365] unable to IoTHubClientCore_LL_UploadToBlob
@ewertons ewertons self-assigned this Dec 19, 2024
@ewertons
Copy link
Contributor

We have tests for verifying this scenario, but here are the results of a manual test:

user@ubuntu1804:/home/user/code/s1/azure-iot-sdk-c/cmake/iothub_client/samples/iothub_client_sample_upload_to_blob_with_retry$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.6 LTS
Release:        18.04
Codename:       bionic
user@ubuntu1804:/home/user/code/s1/azure-iot-sdk-c/cmake/iothub_client/samples/iothub_client_sample_upload_to_blob_with_retry$ git status
HEAD detached at LTS_08_2024
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

        modified:   ../../../../iothub_client/samples/iothub_client_sample_upload_to_blob_with_retry/iothub_client_sample_upload_to_blob_with_retry.c

no changes added to commit (use "git add" and/or "git commit -a")
user@ubuntu1804:/home/user/code/s1/azure-iot-sdk-c/cmake/iothub_client/samples/iothub_client_sample_upload_to_blob_with_retry$ git log -n 1
commit 9c70f98b7b659e169ae44389a5142f3a386c5791 (HEAD, tag: LTS_08_2024, origin/lts_08_2024)
Author: Ewerton Scaboro da Silva <[email protected]>
Date:   Mon Aug 12 15:03:03 2024 -0700

    Update LTS Schedule for LTS_08_2024
user@ubuntu1804:/home/user/code/s1/azure-iot-sdk-c/cmake/iothub_client/samples/iothub_client_sample_upload_to_blob_with_retry$
user@ubuntu1804:/home/user/code/s1/azure-iot-sdk-c/cmake/iothub_client/samples/iothub_client_sample_upload_to_blob_with_retry$ az storage blob delete -c "upload2blob" --name "user-device-x509-1/subdir/hello_world_mb_with_retry.txt"
user@ubuntu1804:/home/user/code/s1/azure-iot-sdk-c/cmake/iothub_client/samples/iothub_client_sample_upload_to_blob_with_retry$ az storage blob exists -c "upload2blob" --name "user-device-x509-1/subdir/hello_world_mb_with_retry.txt"
{
  "exists": false
}
user@ubuntu1804:/home/user/code/s1/azure-iot-sdk-c/cmake/iothub_client/samples/iothub_client_sample_upload_to_blob_with_retry$ ./iothub_client_sample_upload_to_blob_with_retry
Starting the IoTHub client sample upload to blob...
*   Trying 123.456.789.123...
* TCP_NODELAY set
* Connected to user-iothub1.azure-devices.net (123.456.789.123) port 443 (#0)
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: C=US; ST=Washington; L=Redmond; O=Microsoft Corporation; CN=*.azure-devices.net
*  start date: Dec  4 00:00:00 2024 GMT
*  expire date: May  4 23:59:59 2025 GMT
*  subjectAltName: host "user-iothub1.azure-devices.net" matched cert's "*.azure-devices.net"
*  issuer: C=US; O=Microsoft Corporation; CN=MSFT RS256 CA-1
*  SSL certificate verify ok.
> POST /devices/user-device-x509-1/files/?api-version=2016-11-14 HTTP/1.1
Host: user-iothub1.azure-devices.net
Content-Type: application/json
Accept: application/json
User-Agent: iothubclient/1.13.0
Content-Length: 54

* upload completely sent off: 54 out of 54 bytes
< HTTP/1.1 200 OK
< content-length: 428
< content-type: application/json
< date: Thu, 19 Dec 2024 22:23:22 GMT
<
* Connection #0 to host user-iothub1.azure-devices.net left intact
*   Trying 123.456.789.123...
* TCP_NODELAY set
* Connected to user-iothub1.azure-devices.net (123.456.789.123) port 443 (#0)
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: C=US; ST=Washington; L=Redmond; O=Microsoft Corporation; CN=*.azure-devices.net
*  start date: Dec  4 00:00:00 2024 GMT
*  expire date: May  4 23:59:59 2025 GMT
*  subjectAltName: host "user-iothub1.azure-devices.net" matched cert's "*.azure-devices.net"
*  issuer: C=US; O=Microsoft Corporation; CN=MSFT RS256 CA-1
*  SSL certificate verify ok.
> POST /devices/user-device-x509-1/files/notifications/?api-version=2016-11-14 HTTP/1.1
Host: user-iothub1.azure-devices.net
Content-Type: application/json
Accept: application/json
User-Agent: iothubclient/1.13.0
Content-Length: 206

* upload completely sent off: 206 out of 206 bytes
< HTTP/1.1 204 No Content
< date: Thu, 19 Dec 2024 22:23:23 GMT
<
* Connection #0 to host user-iothub1.azure-devices.net left intact
hello world blob has been created
Press any key to continue
user@ubuntu1804:/home/user/code/s1/azure-iot-sdk-c/cmake/iothub_client/samples/iothub_client_sample_upload_to_blob_with_retry$ az storage blob exists -c "upload2blob" --name "user-device-x509-1/subdir/hello_world_mb_with_retry.txt"
{
  "exists": true
}
user@ubuntu1804:/home/user/code/s1/azure-iot-sdk-c/cmake/iothub_client/samples/iothub_client_sample_upload_to_blob_with_retry$ az storage blob download -c "u
pload2blob" --name "user-device-x509-1/subdir/hello_world_mb_with_retry.txt"
Finished[#############################################################]  100.0000%
Hello World from iothub_client_sample_upload_to_blob_with_retry: 0
Hello World from iothub_client_sample_upload_to_blob_with_retry: 1
Hello World from iothub_client_sample_upload_to_blob_with_retry: 2
Hello World from iothub_client_sample_upload_to_blob_with_retry: 3
Hello World from iothub_client_sample_upload_to_blob_with_retry: 4
Hello World from iothub_client_sample_upload_to_blob_with_retry: 5
Hello World from iothub_client_sample_upload_to_blob_with_retry: 6
Hello World from iothub_client_sample_upload_to_blob_with_retry: 7
Hello World from iothub_client_sample_upload_to_blob_with_retry: 8
Hello World from iothub_client_sample_upload_to_blob_with_retry: 9
user@ubuntu1804:/home/user/code/s1/azure-iot-sdk-c/cmake/iothub_client/samples/iothub_client_sample_upload_to_blob_with_retry$

Important points to consider in your setup:

  1. You are using the correct certificate for client authentication, and it's valid and entered correctly.
  2. Please see this temporary branch for reference on how to use the iothub_device_client with x509 logical devices: https://github.com/Azure/azure-iot-sdk-c/blob/ewertons/u2b_x509_sample/iothub_client/samples/iothub_client_sample_upload_to_blob_with_retry/iothub_client_sample_upload_to_blob_with_retry.c#L87
  3. Although you are connecting to the Azure IoT Hub using x509 authentication, the upload to blob part uses SAS tokens when it gets to connect to the Azure Storage, that's mandatory and cannot be changed as per current Azure IoT Hub design.

For reference, this is the storage account configuration we used:

user@ubuntu2404:/home/user/code/s2/azure-iot-sdk-c/cmake/iothub_client/samples/iothub_client_sample_upload_to_blob_with_retry$ az storage account show --name mystorageaccount1
{
  "accessTier": "Hot",
  "accountMigrationInProgress": null,
  "allowBlobPublicAccess": false,
  "allowCrossTenantReplication": false,
  "allowSharedKeyAccess": true,
  "allowedCopyScope": null,
  "azureFilesIdentityBasedAuthentication": null,
  "blobRestoreStatus": null,
  "creationTime": "2024-12-19T19:43:56.981226+00:00",
  "customDomain": null,
  "defaultToOAuthAuthentication": false,
  "dnsEndpointType": "Standard",
  "enableExtendedGroups": null,
  "enableHttpsTrafficOnly": true,
  "enableNfsV3": null,
  "encryption": {
    "encryptionIdentity": null,
    "keySource": "Microsoft.Storage",
    "keyVaultProperties": null,
    "requireInfrastructureEncryption": false,
    "services": {
      "blob": {
        "enabled": true,
        "keyType": "Account",
        "lastEnabledTime": "2024-12-19T19:43:57.559362+00:00"
      },
      "file": {
        "enabled": true,
        "keyType": "Account",
        "lastEnabledTime": "2024-12-19T19:43:57.559362+00:00"
      },
      "queue": null,
      "table": null
    }
  },
  "extendedLocation": null,
  "failoverInProgress": null,
  "geoReplicationStats": null,
  "id": "REDACTED",
  "identity": null,
  "immutableStorageWithVersioning": null,
  "isHnsEnabled": null,
  "isLocalUserEnabled": null,
  "isSftpEnabled": null,
  "isSkuConversionBlocked": null,
  "keyCreationTime": {
    "key1": "2024-12-19T19:43:57.121806+00:00",
    "key2": "2024-12-19T19:43:57.121806+00:00"
  },
  "keyPolicy": null,
  "kind": "StorageV2",
  "largeFileSharesState": "Enabled",
  "lastGeoFailoverTime": null,
  "location": "westus2",
  "minimumTlsVersion": "TLS1_2",
  "name": "mystorageaccount1",
  "networkRuleSet": {
    "bypass": "AzureServices",
    "defaultAction": "Allow",
    "ipRules": [],
    "ipv6Rules": [],
    "resourceAccessRules": null,
    "virtualNetworkRules": []
  },
  "primaryEndpoints": {
	REDACTED
  },
  "primaryLocation": "westus2",
  "privateEndpointConnections": [],
  "provisioningState": "Succeeded",
  "publicNetworkAccess": "Enabled",
  "resourceGroup": "REDACTED",
  "routingPreference": null,
  "sasPolicy": null,
  "secondaryEndpoints": {
	REDACTED
  },
  "secondaryLocation": "westcentralus",
  "sku": {
    "name": "Standard_RAGRS",
    "tier": "Standard"
  },
  "statusOfPrimary": "available",
  "statusOfSecondary": "available",
  "storageAccountSkuConversionStatus": null,
  "tags": {},
  "type": "Microsoft.Storage/storageAccounts"
}
user@ubuntu2404:/home/user/code/s2/azure-iot-sdk-c/cmake/iothub_client/samples/iothub_client_sample_upload_to_blob_with_retry$

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ewertons ewertons

Labels
question
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ewertons @shekharg987