Tukukkk

I need help... I can not access a bucket in my COS that I created with IBM Cloud dashboard from my notebook that I created in Watson Studio. Call this bucket3.

Initially I created a bucket2 from a single f2.zip (csv) file using IBM Watson 'Add Asset' and I am able to access the f2.zip. Call this bucket2.

The f2.zip was uploaded to bucket2 - later found out it was created on a previously COS storage; ie. asset created on IBM Cloud 'cloud-object-storage-nl'. Call this bucket1. I have a single Lite COS on IBM Cloud name = 'cloud-object-storage-xx'.

I can read the f2.zip and I am able to create a new f2.zip in bucket2 using the credential generated (cred_b2_editor) by the "Files UI" button.

IBM Cloud dashboard shows:

bucket1 us-geo Standard

bucket2 us-geo Standard

bucket3 us-east Standard

I am using the example from Creating a new text file in Using Python ibm-cos-sdk.

Fails with "ClientError: An error occurred (413) when calling the PutObject operation: Request Entity Too Large" on 2 cases:

1. when I use endpoint_url = 'endpoints' form the bucket credentials generated by IBM Cloud) - ibm_api_key_id does not matter.

Succeeds: when I use endpoint_url = 'endpoint_url' form the watson credentials generated by Watson Studio... writes to bucket2 regardless of ibm_api_key_id (bucket2 or bucket3)

CODE:

# Point to generated credentials

credDict = dict(b2 = cred_b2_editor,

                b3 = cred_b3_writer,

                watson = cred_watson

                )





bucketName = 'b3'

kwargs = dict(

    ibm_api_key_id=credDict[bucketName]['ibm_api_key_id'],

    ibm_service_instance_id=credDict[bucketName]['cred']['iam_serviceid_crn'], #COS_RESOURCE_CRN,

    ibm_auth_endpoint=COS_AUTH_ENDPOINT,

    config=Config(signature_version="oauth"),

    endpoint_url=credDict[bucketName]['ep_private']

    )

buckName = bucketDict[bucketName].split(':')[-1:][0]



print(buckName, kwargs['ibm_api_key_id'], kwargs['endpoint_url'])

cos = ibm_boto3.resource("s3", **kwargs)



#---> fix: bucketname needed to change with each bucket...

#---> fix: endpoint_url needs to point to private/public endpoint 

cos.Object(buckName, csvBN.replace('.csv','.zip')).put(

                Body=zbuf

            )

Credentials Code - all of the following were generated

'''

Cloud Resource Name or 'bucket ID string'



The last field is the `bucketName`

'''

bucketDict = dict(b2 = 'crn:v1:bluemix:public:cloud-object-storage:global:a/<IDNum>:<serviceID-seperated>:bucket:bucket2',

               b3 = 'crn:v1:bluemix:public:cloud-object-storage:global:a/<IDNum>:<serviceID-seperated>:bucket:bucket3'

              )



# Bucket2 Editor credentials - created by IBM Watson automatically

cred_b2_editor = {

  "apikey": "....",

  "cos_hmac_keys": {

    "access_key_id": "...",

    "secret_access_key": "..."

  },

  "endpoints": "https://cos-service.bluemix.net/endpoints",

  "iam_apikey_description": "Auto generated apikey during resource-key operation for Instance - crn:v1:bluemix:public:cloud-object-storage:global:a/<ServiceIDNum>:<COS_ID>::",

  "iam_apikey_name": "auto-generated-apikey-<apikey_2>",

  "iam_role_crn": "crn:v1:bluemix:public:iam::::serviceRole:Writer",

  "iam_serviceid_crn": "crn:v1:bluemix:public:iam-identity::a/<ServiceIDNum>::serviceid:ServiceId-<serviceID_2>",

  "resource_instance_id": "crn:v1:bluemix:public:cloud-object-storage:global:a/<ServiceIDNum>:<COS_ID>::"

}



## Bucket3 Create via IBM Cloud "New Credentials"

cred_b3_writer = {

  "apikey": "4hEJq-slh28Atvq3XnekZ4YOl0yWiv4LbFigoPS3oiuL",

  "endpoints": "https://cos-service.bluemix.net/endpoints",

  "iam_apikey_description": "Auto generated apikey during resource-key operation for Instance - crn:v1:bluemix:public:cloud-object-storage:global:a/<ServiceIDNum>-<COS_ID>::",

  "iam_apikey_name": "auto-generated-apikey-<apikey_3>",

  "iam_role_crn": "crn:v1:bluemix:public:iam::::serviceRole:Writer",

  "iam_serviceid_crn": "crn:v1:bluemix:public:iam-identity::a/<ServiceIDNum>::serviceid:ServiceId-<ServiceID_3>",

  "resource_instance_id": "crn:v1:bluemix:public:cloud-object-storage:global:a/<ServiceIDNum>:<COS_ID>::"

}



# Created inside juptyer notebook 10/01 button

cred_b2_cos = dict(ibm_api_key_id=cred_b2['apikey'],

                   ibm_auth_endpoint="https://iam.ng.bluemix.net/oidc/token",

                   config=Config(signature_version='oauth'),

                   ep_private='https://s3-api.us-geo.objectstorage.service.networklayer.com',

                   ep_public = 'https://s3-api.us-geo.objectstorage.softlayer.net',

                   cred = cred_b2_editor

                   )

cred_b3_cos = dict(ibm_api_key_id=cred_b3['apikey'],

                   ibm_auth_endpoint="https://iam.ng.bluemix.net/oidc/token",

                   config=Config(signature_version='oauth'),

                   ep_private = 'https://s3.us-east.objectstorage.service.networklayer.com',

                        ep_public = 'https://s3.us-east.objectstorage.softlayer.net',

                   cred = cred_b3_writer

                   )

The solution required:

1. import bucket.configuration.CRN which has bucketname


2. import a 'Writer' ServiceCredential, and


3. setting the bucketName and corresponding kwargs when calling cos.Object().

Key was setting the bucket endpoint_url to the private/public endpoint for the respective bucket.

The code examples were corrected to reflect changes.