Behaviour of EVP_PKEY_get1_RSA in fips mode in opnessl 3.5.6 version

Hi,

I need some help regarding OpenSSL 3.5.6 in FIPS mode.

I have an EVP_PKEY containing an RSA key. When I call:

RSA* rsa = EVP_PKEY_get1_RSA(pkey);

the function returns NULL in FIPS mode.

My questions are:

1. Is EVP_PKEY_get1_RSA() expected to return NULL in FIPS mode with provider-based keys?
   

2. What troubleshooting steps should I perform to determine the root cause?
   

   • How can I verify whether the key is coming from the FIPS provider?
     

   • Are there specific error codes or provider checks I should perform?
     

3. My code currently uses legacy RSA APIs such as:

   RSA_private_encrypt()
   RSA_private_decrypt()

4. What is the recommended replacement for these APIs when running in OpenSSL 3.x and FIPS mode?
   

   • Should I use EVP_PKEY_sign(), EVP_PKEY_decrypt(), EVP_PKEY_encrypt(), or another EVP-based API?
     

   • If the operation is actually being used for raw RSA signing rather than encryption, what is the correct migration path?
     

Additional information:

• OpenSSL version: 3.5.6
  

• FIPS provider enabled
  

• The same code works when FIPS mode is disabled.
  

Any guidance would be appreciated.

Thanks.