SNOW-1462365: ERRPR : PAD BLOCK CORRUPTED => works locally, but not when deployed to k8s pod.

[gautam-goudar]

Please answer these questions before submitting your issue.
In order to accurately debug the issue this information is required. Thanks!

1. What version of .NET driver are you using?
   Snowflake.Data - 3.1.0

2. What operating system and processor architecture are you using?
   macOS Sonoma Version 14.5

3. What version of .NET framework are you using?
   .net core 7.0.314

4. What did you do?

I have the below code to connect to Snowflake and run a query

   SnowflakeDbConnectionStringBuilder connStringBuilder = new()
        {
            ["ACCOUNT"] = _account,
            ["USER"] = _user,
            ["AUTHENTICATOR"] = "SNOWFLAKE_JWT",
            ["PRIVATE_KEY"] = _keyFileContent,
            ["PRIVATE_KEY_PWD"] = _keyPassPhrase,
            ["ROLE"] = "ENGINEERING_READ_ONLY",
            ["WAREHOUSE"] = _warehouse
        };
        
    using var connection = new SnowflakeDbConnection();
    connection.ConnectionString = connStringBuilder.ConnectionString;

    await connection.OpenAsync();
   
    using var command = connection.CreateCommand();
    command.CommandText = query;

    using var reader = await command.ExecuteReaderAsync();

    DataTable dt = new();
    dt.Load(reader);

    return (dt);

5. What did you expect to see?

Running locally, using VS Code, I am able to connect and fetch the results successfully as expected.

However, when the app is deployed to k8s, I see the below error, when the same certificate file (or it's contents) are used to
query Snowflake

NZO-625: Error System.AggregateException: One or more errors occurred. (Error: Snowflake Internal Error: Unable to connect SqlState: 08006, VendorCode: 270001, QueryId: )
---> Snowflake.Data.Client.SnowflakeDbException (0x80004005): Error: Snowflake Internal Error: Unable to connect SqlState: 08006, VendorCode: 270001, QueryId: 
---> System.AggregateException: One or more errors occurred. (One or more errors occurred. (Error: Could not read private key with value passed in connection string. \n Error : incorrect private key value or private key format: use "\n" for newlines and double the equals sign. SqlState: , VendorCode: 270052, QueryId: ))
---> System.AggregateException: One or more errors occurred. (Error: Could not read private key with value passed in connection string. \n Error : incorrect private key value or private key format: use "\n" for newlines and double the equals sign. SqlState: , VendorCode: 270052, QueryId: )
---> Snowflake.Data.Client.SnowflakeDbException (0x80004005): Error: Could not read private key with value passed in connection string. \n Error : incorrect private key value or private key format: use "\n" for newlines and double the equals sign. SqlState: , VendorCode: 270052, QueryId: 
---> Org.BouncyCastle.OpenSsl.PemException: problem creating ENCRYPTED private key: Org.BouncyCastle.Crypto.InvalidCipherTextException: pad block corrupted

I have tried using both "PRIVATE_KEY" and "PRIVATE_KEY_FILE" and they both result in the pad block corrupted error when running from k8s.

I used the below docker ADD command to copy the p8 file during the application deployment. Both COPY and ADD end up in the same above error

ADD ./Deployments/rsa_${ASPNETCORE_ENVIRONMENT}_key.p8 ./rsa_${ASPNETCORE_ENVIRONMENT}_key.p8

6. Can you set logging to DEBUG and collect the logs?

   https://community.snowflake.com/s/article/How-to-generate-log-file-on-Snowflake-connectors

   There is an example in READMD.md file showing you how to enable logging.

7. What is your Snowflake account identifier, if any? (Optional)

[sfc-gh-dszmolka]

hi and thanks for raising this issue with us, will take a look

[gautam-goudar]

I was able to find the root cause finally. The problem is with the way a passphrase is read locally and in k8s by the same code base.

What was shared with me was

[redacted]v\"qS[redacted]

The actual working one is (without the \ )

[redacted]v"qS[redacted]

I suppose the \ interpretation when running locally vs in k8s is different. Locally, the code is able to understand that \ is required for the = character in the passphrase.

But, what was very mis-leading was the error that was being displayed by the library

\n Error : incorrect private key value or private key format: use "\n" for newlines and double the equals sign. SqlState: , VendorCode: 270052, QueryId: )

Thanks.

[sfc-gh-dszmolka]

glad you figured this out and even shared the solution. apologies, did not have the time yet to set up a repro, you were much quicker than that ;)
guess we can enhance the error message to provide more useful pointers.