Snapshot repository certificate error with Oracle Cloud S3

Hi all,

I am currently trying to use OCI Object storage for repository in Crate 5.2.3 and I am facing a certificane common name error.

When adding the repository with CREATE REPOSITORY sentence I get the following error:

RepositoryVerificationException[[bemp_dev_s3_repo] Unable to verify the repository, [bemp_dev_s3_repo] is not accessible on master node:
SdkClientException 'Unable to execute HTTP request: Certificate for <bucket-name.namespace.compat.objectstorage.eu-frankfurt-1.oraclecloud.com> doesn't match any of the subject alternative names: [swiftobjectstorage.eu-frankfurt-1.oraclecloud.com]']

Looking in the internet I have found tha for OCI Object storage to work the client needs to use SNI java - How to config Oracle cloud certificate? - Stack Overflow

Does Crate S3 client support SNI?

Best regards,

Dear @iames,

thank you for writing in.

Does Crate S3 client support SNI?

I would not know how it would work otherwise for all the S3 buckets out there, as all the user-specific alias names will probably hit the same server configuration, where virtual hosts are dispatched, well, by their host names, using SNI.

May I ask you to share the specific value of bucket-name.namespace.compat.objectstorage.eu-frankfurt-1.oraclecloud.com with us? If you think it’s too sensitive, don’t hesitate to shoot a private message at me. Thanks!

With kind regards,
Andreas.

I am sending that information via private message since namespace is unique for the tenant, used for other services and cannot be changed. Thank you.

1 Like

Hi @iames.

Thank you. I am responding here with an anonymized variant.

So, the problem is that accessing the HTTP resource on a bucket level fails, because of certificate SAN vs. hostname mismatch error.

Certificate for <bucket.namespace.compat.objectstorage.eu-frankfurt-1.oraclecloud.com> doesn't match any of the subject alternative names: [swiftobjectstorage.eu-frankfurt-1.oraclecloud.com]

However, we discovered that it works on the namespace level, i.e. namespace.compat.objectstorage.eu-frankfurt-1.oraclecloud.com does have a valid certificate, because it is the wildcard SSL certificate for *.compat.objectstorage.eu-frankfurt-1.oraclecloud.com.

A wildcard SSL certificate typically covers all subdomains of a domain, but it does not cover arbitrary labels or path components in the URL. From this, I am figuring that you would need to acquire a dedicated SSL certificate for accessing the resource on the bucket level. Oracle’s Cloud Console should offer an opportunity for that.

Please let us know if this helps, or if you think some other parts are co-responsible for the problem.

With kind regards,
Andreas.

Thank you Andreas.

I have been reading Oracle Object Storage documentation more carefully and in the S3 compatibility section in " Modifying your application" the fourth item says:

  1. Use path-based access in your application. Virtual host-style access (accessing a bucket as {bucketnamespace}.compat.objectstorage.{region}.oraclecloud.com) is not supported.

I think this is the problem. I am not an expert in S3, but is Crate accessing using ‘virtual host-style’? Is there any way to change this behavior?

Best regards,

Hi again,

apologies for the delayed response. I’ve just created a corresponding report on the issue tracker at `CREATE REPOSITORY`: Compatibility with Oracle Cloud S3 · Issue #14177 · crate/crate · GitHub.

If you are on GitHub, you may want to subscribe on this issue, in order to get notified about any progress. Other than this, we will also respond to this discussion.

Thanks again for the report!

With kind regards,
Andreas.

I have been working on a solution for this and I have finally modified Crate code to include a new param ‘use_path_style_access’ in the S3 repository definition. This parameter forces the call to enablePathStyleAccess() in the AmazonS3ClientBuilder.

Tested with Oracle Cloud and it works as expected.

Is quite a simple change, what is the best way to contribute this change? Can I upload a diff file here?

1 Like

Thank you for your efforts, the best would be if you could raise a pull request for review in the crate/crate repo, please take a look at the
contributing guide.

I signed the CLA (ICLA) some days ago and I haven’t received any new notification. Do I have permission to create a new PR? Do I need to wait more? I know we are approaching complicated dates…

Hi, apologies for the delay on this, I just checked internally and confirmed we got your signed ICLA, feel free to go ahead raising the PR.

I have tried to upload a new branch to Crate repository but permission denied. Am I missing something?

You can’t directly commit a branch to the crate repository. You need to fork the repository and make your changes there in a commit. Then you can create a pull request to merge the changes into the crate repository.

I have finally managed to create de PR #15222, I hope everything is correct. Sorry for my ignorance.

2 Likes

Thanks for your efforts. PR has been merged and will be available in CrateDB 5.7.0

Crossposting for the record: `CREATE REPOSITORY`: Compatibility with Oracle Cloud S3 · Issue #14177 · crate/crate · GitHub

1 Like