Abstract: This article will take you through the two common authentication methods of OBS: header-borne signature and URL-borne signature.
OBS provides Representational State Transfer (REST) -style apis that you can invoke through HTTP or HTTPS requests. Before invoking the OBS API, understand the OBS authentication mode. This article takes you through two common authentication methods for OBS — header-borne signatures and URL-borne signatures.
1. Carry the signature calculation in the Header
Website links: support.huaweicloud.com/api-obs/obs…
1.1 Calculation principle and method of signature
Principle of graphic
Calculation method
(1). Construct request string (StringToSign).
The request string is constructed as follows:
StringToSign =
HTTP-Verb + “\n” +
Content-MD5 + “\n” +
Content-Type + “\n” +
Date + “\n” +
CanonicalizedHeaders + CanonicalizedResource
(2) utF-8 encoding is performed for the results of the first step.
(3). Use SK to calculate hMAC-SHA1 signature for the results of the second step.
(4). Base64 encoding is performed on the results of the third step to obtain the signature.
The signature is in the following form (a base64-encoded string of 28 bits) :
JONydLd9zpf+Eu3IYiUjNmukHN0=
Copy the codeCalculation example
For example, to obtain the object ACL of the object log.conf under bucket obs-test, how to construct the request and calculate the signature?
1. Specify the fields of StringToSign:
Request method: GET;
Request MD5: null
The content-type: empty
Request time: Tue, 28 Jul 2020 06:29:47 GMT
Custom header: empty
Canonical resource :/obs-test/log.conf? acl
Construct request string StringToSign as follows:
StringToSign = "GET Tue, 28 Jul 2020 06:29:47 GMT /obs-test/log.conf? The acl ' ' 'Copy the code3. According to the signature algorithm, StringToSign is calculated by HMAC-SHA1 and BASE64 encoded to obtain the signature result: xYlcrwT9jSaCtY0OnBE01OBR+aA=
1.2. Implementation of signature calculation
For your reference, use Python to calculate the signature code:
Import hashlib import hMAC import binASCII from datetime import datetime # Verify information AK = 'your access_key_id' SK = 'Your secret_access_key_id' # specifies the HTTP method, Optional GET/PUT/DELETE/POST/OPTIONS httpMethod = "GET" # of the specified request Header: the content-type and Content - MD5 contentType = "" conten * * 5 =" " # use dateTime library to generate time, Date = datetime.utcnow().strftime('%a, ') %d %b %Y %H:%M:%S GMT') # canonicalizedHeaders # canonicalizedHeaders = "x-obs-acl:public-read" # canonicalizedHeaders = "x-obs-acl:public-read\n"+'x-obs-storage-class:WARM\n' canonicalizedHeaders =" CanonicalizedResource = "/BucketName/ObjectName" # CanonicalizedResource = "/BucketName/ObjectName? Acl "# CanonicalizedResource = "/" CanonicalizedResource = "/ + "\n" + conten**5 + "\n" + contentType + "\n" + date + "\n" + canonicalizedHeaders + CanonicalizedResource # Hashed = hmac.new(sk.encode (' utF-8 '), canonical_string.encode(' utF-8 '), Hashlib.sha1) encode_canonical = binascii.b2a_base64(hashed.digest())[:-1].decode(' utF-8 ') # Print StringToSign for validation if a problem occurs Print (canonical_string)Copy the codeC signature algorithm example:
Please refer to the obs-community.obs.cn-north-1.myhuaweicloud.com/sign/signat… , download the C language signature calculation code example, where:
The interface for calculating the signature is contained in the sign.h header file.
Sample code for calculating the signature is in the main.c file.
Visual signature calculation tool:
You can also calculate signatures using the visual signature calculation tool provided by OBS.
Tool links:
Obs-community.obs.cn-north-1.myhuaweicloud.com/sign/header…
Description:
1. CanonicalizedHeaders: indicates the OBS header field in the HTTP request header field. That is, x-obs- is used as the header field, for example, X-obs-date, X-obs-acl, x-obs-meta-*.
A. All characters of the keywords in the request header field need to be lowercase. If multiple fields need to be added, sort all the fields in ascending order according to the keyword lexicographical order.
B. If the request header fields have the same name, merge them. For example, x-obs-meta-name:name1 and x-obs-meta-name:name2, separate the values of the fields with the same name (name1 and name2) with commas (,) and merge them into x-obs-meta-name:name1,name2.
C. The keywords in the request header field in the header field cannot contain non-ASCII characters or unrecognizable characters. It is not recommended to use non-ASCII or unrecognizable characters in the request header field. If non-ASCII or unrecognizable characters must be used, the client needs to do the encoding and decoding by itself, which can be URL encoding or Base64 encoding. The server does not do the decoding.
D. Discard the request header field that contains a meaningless space or table key. For example, x-obs-meta-name:name (name preceded by a meaningless space) needs to be converted to x-obs-meta-name:name.
E. Each request header field should end with a new line.
2. CanonicalizedResource indicates the OBS resource specified in the HTTP request, which is constructed as follows:
< bucket name + Object name >+[subresource]…
A. Access OBS using the self-defined domain name of the bucket binding. The bucket name is /obs.ccc.com/object, where obs.ccc.com is the self-defined domain name of the bucket binding. If there is no object name, such as enumerating buckets, “/obs.ccc.com/”;
B. If OBS is accessed without the user-defined domain name bound to the bucket, the value is /bucket/object. If no object name exists, for example, /bucket/. If the bucket name does not exist, the value is /.
C. If there are sub-resources, add them. For example,? The acl,? Logging.
3. If you need to use the temporary AK/SK+SecurityToken to calculate the signature, use the same method to calculate the signature, but add x-obs-security-token:… to the header field. Field.
4. See the description at the end of this article for the method of calculating content-MD5.
5. The codes for calculating signatures in other languages can be found in the SDK of corresponding languages.
Support.huaweicloud.com/sdkreferenc…
1.3 common Problems
1. Error: Signature Does Not Match OBS
There are two possible cases of signature mismatch:
A. You do not use the correct AK/SK. You can check whether the SK used to calculate the signature and the AK carried when sending the request are correct and match.
B. The StringToSign constructed when you compute the signature does not match the StringToSign calculated by the server based on the HTTP request received. You can check the StringToSign returned by the server and compare it to the StringToSign used by the local computed signature.
The following figure shows the StringToSign returned by the server as restored by receiving the HTTP request. You can analyze why your signature calculation failed by comparing your local StringToSign with the HTTP request you sent to the server.
2. OBS: Request has expired
In this case, please check whether the Date you carry is correct. To ensure the timeliness of the request, the time difference between the Date header field you carry and the server must be within 15 minutes (UTC time on the server). If you carry the X-OBS-date header field, Check whether the time difference between the X-obs-date and the server is within 15 minutes.
2. Carry the signature in the URL
OBS service supports users to construct a URL for a specific operation, which contains information such as user AK, signature, validity period and resource. Anyone who gets the URL can perform the operation. OBS service considers that the request is performed by the user who issued the URL after receiving the request. For example, a URL is constructed to download an object with a signature. The person receiving the URL can download the object, but the URL is only valid for an expiration time specified in Expires. The signature carried in the URL is mainly used to enable others to authenticate themselves with the pre-signed URL and perform predefined operations without providing others with the Secret Access Key.
Website links support.huaweicloud.com/api-obs/obs…
2.1 Calculation principle and method of signature
Principle of graphic
Calculation method
1. Construct request string (StringToSign).
The request string is constructed as follows:
StringToSign =
HTTP-Verb + “\n” +
Content-MD5 + “\n” +
Content-Type + “\n” +
Date + “\n” +
CanonicalizedHeaders + CanonicalizedResource
2. Utf-8 encoding is performed for the result of the first step.
3. Use SK to calculate the HMAC-SHA1 signature.
4. Base64 encoding is performed on the result of the third step to obtain the signature.
The signature is in the following form (a base64-encoded string of 28 bits) :
JONydLd9zpf+Eu3IYiUjNmukHN0=
The signature calculation method carried in the URL is the same as the signature calculation method carried in the Header, but the Date must be replaced with the UNIX timestamp.
The URL with the signature looks like this:
obs-ycytest.obs.cn-north-1.myhuaweicloud.com/?AccessKeyId=YN97UCJEKF2ALJ44AHAN&Expires=1575452568&Signature=0wG/GF7XgmOa tCFhwHJh0J6NrtQ=Copy the codeIts corresponding StringToSign is
GET
1575452568
/obs-ycytest/
Copy the codeThe specific meanings of parameters carried in URL are shown in the following table:
Calculation example
For example, to obtain the object ACL of the object log.conf under bucket obs-test, how to construct the request and calculate the signature?
1. Specify the fields of StringToSign:
Request method: GET; MD5: empty Content-type: empty request time: 1595918661 CanonicalizedHeaders: CanonicalizedResource :/obs-test/log.conf? aclCopy the codeConstruct request string StringToSign as follows:
StringToSign = "" GET 1595918661 /obs-test/log.conf? The acl ' ' 'Copy the code3. According to the signature algorithm, StringToSign is calculated by HMAC-SHA1 and BASE64 encoded to obtain the signature result: lLcYw1fFMJv5m+MS0XenNrqJlag=
According to the calculation results, the URL with signature can be generated by stitching together the URL as follows:
obs-test.obs.myhuaweicloud.com/log.conf?AccessKeyId=xxx&acl&Expires=1595918661&Signature= lLcYw1fFMJv5m+MS0XenNrqJlag=
2.2. Implementation of signature calculation
When carrying a signature in a URL, simply replace Date with a UNIX timestamp to calculate the corresponding signature, so the corresponding code will not be described.
If you need to use temporary AK/SK+SecurityToken to calculate the signature, the method of calculating the signature remains the same, but you need to add? To the corresponding CanonicalizedResource. X-ray obs ws-security – token =…” Field, and the calculated signature must be URL encoded. The code for calculating a signature using a temporary AK/SK+SecurityToken is as follows:
import hashlib import hmac import binascii import urllib.request AK = 'Input Your AccessKeyId' SK = 'Input Your SecretKeyId' Token = 'Input Your SecurityToken' httpMethod = "GET" contentType = "" Conten**5 = '' date = '1594972984' canonicalizedHeaders = '' CanonicalizedResource = "/messageflow/flowengine.tar.gz" + "? x-obs-security-token=" + Token canonical_string = httpMethod + "\n" + Conten**5 + "\n" + contentType + "\n" + date + "\n" + canonicalizedHeaders + CanonicalizedResource hashed = hmac.new(SK.encode('UTF-8'), canonical_string.encode('UTF-8'), hashlib.sha1) encode_canonical = binascii.b2a_base64(hashed.digest())[:-1].decode('UTF-8') url= 'messageflow.obs.myhuaweicloud.com/flowengine.tar.gz?x-obs-security-token={}&Expires={}&AccessKeyId={}&Signature={}'.for mat(Token, date, AK, urllib.request.quote(encode_canonical)) print(url)Copy the codeDescription:
1. When calculating a signature, Date represents a UNIX timestamp.
2. Do not use content-MD5, Content-Type, or CanonicalizedHeaders to calculate the signature if you want to use the predefined URL that carries the signature in the URL in the browser. Otherwise, the browser cannot carry these parameters. After the request is sent to the server, a signature error message is displayed.
3. Calculation method of Content-MD5
3.1 Calculation method of Content-MD5
Take message “0123456789” as an example. The following describes how to calculate the content-MD5 of the string.
1. Calculate the binary array (128 bits) encrypted by MD5.
2. Base64 encodes this binary array (instead of 32-bit strings).
Take Python as an example:
>>> import base64,hashlib >>> hash = hashlib.md5() >>> hash.update("0123456789".encode(' UTF-8 ')) >>> Base64.b64encode (Hash.digest ()) 'eB5eJF1ptWaXm4bijSPyxw==' Note: hash.digest() computs the binary array (128 bits). >>> hash.digest() 'x\x1e^$]i\xb5f\x97\x9b\x86\xe2\x8d#\xf2\xc7'Copy the code3.2 Implementation of Content-MD5 calculation
The MD5 code of the Python calculation file is used as an example.
import os import base64 import hashlib def md5_file_encode_by_size_offset(file_path=None, size=None, offset=None, chuckSize=None): if file_path is not None and size is not None and offset is not None: m = hashlib.md5() with open(file_path, 'rb') as fp: CHUNKSIZE = 65536 if chuckSize is None else chuckSize fp.seek(offset) read_count = 0 while read_count < size: read_size = CHUNKSIZE if size - read_count >= CHUNKSIZE else size - read_count data = fp.read(read_size) read_count_once = len(data) if read_count_once <= 0: break m.update(data) read_count += read_count_once return base64.b64encode(m.digest()).decode() file_path = r'Input Your File Path' size = os.path.getsize(file_path) Conten**5 = md5_file_encode_by_size_offset(file_path=file_path, size=size, offset=0) print(Conten**5)Copy the code3.3 Frequently Asked Questions
A common mistake is to base64 encoding a calculated 32-bit string directly.
# hash.hexdigest(), which evaluates to the visible 32-bit string encoding. >>> import base64,hashlib >>> hash = hashlib.md5() >>> hash.update("0123456789".encode(' UTF-8 ')) >>> hash.hexdigest() '781e5e245d69b566979b86e28d23f2c7'Copy the codeError MD5 value base64 encoding result:
>>> base64.b64encode(hash.hexdigest())
'NzgxZTVlMjQ1ZDY5YjU2Njk3OWI4NmUyOGQyM2YyYzc='
Copy the codeClick to follow, the first time to learn about Huawei cloud fresh technology ~