Win32 Application CryptoAPI

D

Darren Bennett

Hi There,

I have been scanning the newsgroups for a solution to my problem and have
found that a few others are also experiencing the same problem but none of
the solutions provided to them seem to work for me.

I have a native Win32 application (written in C++) that needs to encrypt
some data, pass that data to a web service (written in C#) for decryption. I
just can't seem to decrypt the data correctly.

To demonstrate the problem, I have written a native Win32 (C++) application
to encrypt some data and write the encrypted data out to disk:

HCRYPTPROV hProv;
HCRYPTHASH hHash;
HCRYPTKEY hKey;
HANDLE hFile;
DWORD dwNumBytes, dwBytesWritten;
BYTE byPassword[] = {65, 66, 67, 68, 69, 70}; // equals "ABCDEF"
BYTE byData[] = {10, 20, 30, 40, 50, 60, 70, 80, 0, 0, 0, 0, 0, 0, 0,
0};

// Create the Cryptograhic Provider Object
if (CryptAcquireContext(&hProv, NULL, MS_DEF_PROV, PROV_RSA_FULL,
CRYPT_VERIFYCONTEXT)) {
// Create a hash object
if (CryptCreateHash(hProv, CALG_MD5, 0, 0, &hHash)) {
// Hash in the password
if (CryptHashData(hHash, byPassword, sizeof(byPassword), 0)) {
// Derive a session key from the hash object.
if (CryptDeriveKey(hProv, CALG_RC2, hHash, 0, &hKey)) {
dwNumBytes = 8;
if (CryptEncrypt(hKey, NULL, TRUE, 0, byData, &dwNumBytes,
sizeof(byData))) {
// Create a file to store the encrypted data
if ((hFile = CreateFile(_T("C:\\Encrypted.dat"), GENERIC_WRITE, 0,
NULL, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL)) != INVALID_HANDLE_VALUE) {
// Write the encrypted data out to file
WriteFile(hFile, &byData, dwNumBytes, &dwBytesWritten, NULL);
// Close the file handdle
CloseHandle(hFile);
}
}
// Destroy the session key
CryptDestroyKey(hKey);
}
}
// Destroy the hash object
CryptDestroyHash(hHash);
}
// Release the cryptographic provider context
CryptReleaseContext(hProv, 0);
}

I have also written a .NET C# application to read the data from file and try
and decrypt the data:

int iNumBytes;
FileStream fsData;
byte[] byData = new byte[16];
byte[] bySalt = {0,0,0,0,0,0,0,0};
byte[] byInitVect = {0,0,0,0,0,0,0,0};

// Open the file containing the encrypted data, read inb the data and close
the file
fsData = File.Open("C:\\Encrypted.dat", FileMode.Open);
iNumBytes = fsData.Read(byData, 0, byData.Length);
fsData.Close();

// Derive a session key from the password using an MD5 hash
PasswordDeriveBytes SessionKey = new PasswordDeriveBytes("ABCDEF", bySalt,
"MD5", 1);

// Set up an RC2 cryptographic object
RC2CryptoServiceProvider Rc2 = new RC2CryptoServiceProvider();
Rc2.Mode = CipherMode.CBC;
Rc2.KeySize = 40;
Rc2.EffectiveKeySize = 40;
Rc2.BlockSize = 64;
Rc2.Key = SessionKey.GetBytes(5);

// Decrypt the data
ICryptoTransform decryptor = Rc2.CreateDecryptor(SessionKey.GetBytes(8),
byInitVect);
byte[] myOutputBytes = new byte[decryptor.OutputBlockSize];
iNumBytes = decryptor.TransformBlock(byData, 0, decryptor.InputBlockSize,
myOutputBytes, 0);

The problem is that the data just does not decrypt. I really don't care what
type of hashing or encryption algorithms are used so long as they are
supported on Win98/Me and NT4.0SP6/2000/XP/2003.

Any help in solving this problem would be greatly appreciated.

Thanks,

-Darren-
 
D

Doug Barlow

Darren,

I haven't tried running your programs yet, but I do have a few suggestions.

First, you've probably chosen the worst case algorithm for compatibility --
The old Microsoft CSPs mess with 40-bit RC2 to try to make it stronger,
giving it an 88-byte salt, so you've got to manage the salt values, too.
You might have better luck moving up to a 128-bit RC2 key, which doesn't
default to a special salt value. Take a look at the MSDN documentation for
CryptDeriveKey for details.

Next, I wouldn't assume the .NET PasswordDeriveBytes class derives the same
key as the Win32 CryptDeriveKey service. In fact, I'd bet against it. You
should come up with a common method for deriving a key, and make sure you
use the same key and salt in both applications. You can import a plain-text
session key into CryptoAPI by following the example shown in
http://support.microsoft.com/default.aspx?scid=kb;en-us;228786.

I hope that helps,

Doug Barlow
The Soft Pedal Shop
http://www.softpedal.net
 
D

Darren Bennett

Hi Doug,

Thanks for your reply to my question.

I have modified the code to make it 128bit RC2 encryption as you suggested
but I still end up with the same result. I have included the updated code
below:

Win32 Application:
============
HCRYPTPROV hProv;
HCRYPTHASH hHash;
HCRYPTKEY hKey;
HANDLE hFile;
DWORD dwNumBytes, dwBytesWritten;
BYTE byPassword[] = {65, 66, 67, 68, 69, 70}; // equals "ABCDEF"
BYTE byData[] = {10, 20, 30, 40, 50, 60, 70, 80, 0, 0, 0, 0, 0, 0,
0, 0};

// Create the Cryptograhic Provider Object
if (CryptAcquireContext(&hProv, NULL, MS_ENHANCED_PROV, PROV_RSA_FULL,
CRYPT_VERIFYCONTEXT)) {
// Create a hash object
if (CryptCreateHash(hProv, CALG_MD5, 0, 0, &hHash)) {
// Hash in the password
if (CryptHashData(hHash, byPassword, sizeof(byPassword), 0)) {
// Derive a session key from the hash object.
if (CryptDeriveKey(hProv, CALG_RC2, hHash, CRYPT_NO_SALT, &hKey)) {
dwNumBytes = 8;
if (CryptEncrypt(hKey, NULL, TRUE, 0, byData, &dwNumBytes,
sizeof(byData))) {
// Create a file to store the encrypted data
if ((hFile = CreateFile(_T("C:\\Encrypted.dat"), GENERIC_WRITE,
0, NULL, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL)) !=
INVALID_HANDLE_VALUE) {
// Write the encrypted data out to file
WriteFile(hFile, &byData, dwNumBytes, &dwBytesWritten, NULL);
// Close the file handdle
CloseHandle(hFile);
}
}
// Destroy the session key
CryptDestroyKey(hKey);
}
}
// Destroy the hash object
CryptDestroyHash(hHash);
}
// Release the cryptographic provider context
CryptReleaseContext(hProv, 0);
}

C# Application:
==========
int iNumBytes;
FileStream fsData;
byte[] byData = new byte[16];
byte[] byInitVect = {0,0,0,0,0,0,0,0};

// Open the file containing the encrypted data, read inb the data and close
the file
fsData = File.Open("C:\\Encrypted.dat", FileMode.Open);
iNumBytes = fsData.Read(byData, 0, byData.Length);
fsData.Close();

// Derive a session key from the password using an MD5 hash
PasswordDeriveBytes SessionKey = new PasswordDeriveBytes("ABCDEF", null,
"MD5", 1);

// Set up an RC2 cryptographic object
RC2CryptoServiceProvider Rc2 = new RC2CryptoServiceProvider();
Rc2.Mode = CipherMode.CBC;
Rc2.KeySize = 128;
Rc2.EffectiveKeySize = 128;
Rc2.BlockSize = 64;

// Decrypt the data
ICryptoTransform decryptor = Rc2.CreateDecryptor(SessionKey.GetBytes(16),
byInitVect);
byte[] myOutputBytes = new byte[decryptor.OutputBlockSize];
iNumBytes = decryptor.TransformBlock(byData, 0, decryptor.InputBlockSize,
myOutputBytes, 0);


Any help would be greatly appreciated.

Thanks,

- Darren -
 
J

Joe Kaplan \(MVP - ADSI\)

You are still making the assumption that PasswordDeriveBytes is going to
yield the same key as your CryptCreateHash/CryptHashData thing is going to
return the same key. Even if they use the same routine under the hood, you
have to remember that strings in .NET are Unicode and your unmanged routine
uses ASCII, so those shouldn't produce the same data.

You might start by hard-coding a key and going from there.

Joe K.

Darren Bennett said:
Hi Doug,

Thanks for your reply to my question.

I have modified the code to make it 128bit RC2 encryption as you suggested
but I still end up with the same result. I have included the updated code
below:

Win32 Application:
============
HCRYPTPROV hProv;
HCRYPTHASH hHash;
HCRYPTKEY hKey;
HANDLE hFile;
DWORD dwNumBytes, dwBytesWritten;
BYTE byPassword[] = {65, 66, 67, 68, 69, 70}; // equals "ABCDEF"
BYTE byData[] = {10, 20, 30, 40, 50, 60, 70, 80, 0, 0, 0, 0, 0, 0,
0, 0};

// Create the Cryptograhic Provider Object
if (CryptAcquireContext(&hProv, NULL, MS_ENHANCED_PROV, PROV_RSA_FULL,
CRYPT_VERIFYCONTEXT)) {
// Create a hash object
if (CryptCreateHash(hProv, CALG_MD5, 0, 0, &hHash)) {
// Hash in the password
if (CryptHashData(hHash, byPassword, sizeof(byPassword), 0)) {
// Derive a session key from the hash object.
if (CryptDeriveKey(hProv, CALG_RC2, hHash, CRYPT_NO_SALT, &hKey)) {
dwNumBytes = 8;
if (CryptEncrypt(hKey, NULL, TRUE, 0, byData, &dwNumBytes,
sizeof(byData))) {
// Create a file to store the encrypted data
if ((hFile = CreateFile(_T("C:\\Encrypted.dat"), GENERIC_WRITE,
0, NULL, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL)) !=
INVALID_HANDLE_VALUE) {
// Write the encrypted data out to file
WriteFile(hFile, &byData, dwNumBytes, &dwBytesWritten, NULL);
// Close the file handdle
CloseHandle(hFile);
}
}
// Destroy the session key
CryptDestroyKey(hKey);
}
}
// Destroy the hash object
CryptDestroyHash(hHash);
}
// Release the cryptographic provider context
CryptReleaseContext(hProv, 0);
}

C# Application:
==========
int iNumBytes;
FileStream fsData;
byte[] byData = new byte[16];
byte[] byInitVect = {0,0,0,0,0,0,0,0};

// Open the file containing the encrypted data, read inb the data and
close
the file
fsData = File.Open("C:\\Encrypted.dat", FileMode.Open);
iNumBytes = fsData.Read(byData, 0, byData.Length);
fsData.Close();

// Derive a session key from the password using an MD5 hash
PasswordDeriveBytes SessionKey = new PasswordDeriveBytes("ABCDEF", null,
"MD5", 1);

// Set up an RC2 cryptographic object
RC2CryptoServiceProvider Rc2 = new RC2CryptoServiceProvider();
Rc2.Mode = CipherMode.CBC;
Rc2.KeySize = 128;
Rc2.EffectiveKeySize = 128;
Rc2.BlockSize = 64;

// Decrypt the data
ICryptoTransform decryptor = Rc2.CreateDecryptor(SessionKey.GetBytes(16),
byInitVect);
byte[] myOutputBytes = new byte[decryptor.OutputBlockSize];
iNumBytes = decryptor.TransformBlock(byData, 0, decryptor.InputBlockSize,
myOutputBytes, 0);


Any help would be greatly appreciated.

Thanks,

- Darren -



Doug Barlow said:
Darren,

I haven't tried running your programs yet, but I do have a few
suggestions.

First, you've probably chosen the worst case algorithm for
compatibility --
The old Microsoft CSPs mess with 40-bit RC2 to try to make it stronger,
giving it an 88-byte salt, so you've got to manage the salt values, too.
You might have better luck moving up to a 128-bit RC2 key, which doesn't
default to a special salt value. Take a look at the MSDN documentation
for
CryptDeriveKey for details.

Next, I wouldn't assume the .NET PasswordDeriveBytes class derives the
same
key as the Win32 CryptDeriveKey service. In fact, I'd bet against it.
You
should come up with a common method for deriving a key, and make sure you
use the same key and salt in both applications. You can import a
plain-text
session key into CryptoAPI by following the example shown in
http://support.microsoft.com/default.aspx?scid=kb;en-us;228786.

I hope that helps,

Doug Barlow
The Soft Pedal Shop
http://www.softpedal.net
Click to expand...
Click to expand...
 
D

Darren Bennett

Thanks for your feedback.

I just couldn't get this working regardless of what I tried, so I have
returned to good old Win32 and P/Invoked the required Win32 Crypto functions
and it works like a treat. It's a shame because I really wanted to get this
running in pure .NET environment.

Thanks everyone for your feedback on this problem.

- Darren -

Joe Kaplan (MVP - ADSI) said:
You are still making the assumption that PasswordDeriveBytes is going to
yield the same key as your CryptCreateHash/CryptHashData thing is going to
return the same key. Even if they use the same routine under the hood, you
have to remember that strings in .NET are Unicode and your unmanged routine
uses ASCII, so those shouldn't produce the same data.

You might start by hard-coding a key and going from there.

Joe K.

Darren Bennett said:
Hi Doug,

Thanks for your reply to my question.

I have modified the code to make it 128bit RC2 encryption as you suggested
but I still end up with the same result. I have included the updated code
below:

Win32 Application:
============
HCRYPTPROV hProv;
HCRYPTHASH hHash;
HCRYPTKEY hKey;
HANDLE hFile;
DWORD dwNumBytes, dwBytesWritten;
BYTE byPassword[] = {65, 66, 67, 68, 69, 70}; // equals "ABCDEF"
BYTE byData[] = {10, 20, 30, 40, 50, 60, 70, 80, 0, 0, 0, 0, 0, 0,
0, 0};

// Create the Cryptograhic Provider Object
if (CryptAcquireContext(&hProv, NULL, MS_ENHANCED_PROV, PROV_RSA_FULL,
CRYPT_VERIFYCONTEXT)) {
// Create a hash object
if (CryptCreateHash(hProv, CALG_MD5, 0, 0, &hHash)) {
// Hash in the password
if (CryptHashData(hHash, byPassword, sizeof(byPassword), 0)) {
// Derive a session key from the hash object.
if (CryptDeriveKey(hProv, CALG_RC2, hHash, CRYPT_NO_SALT, &hKey)) {
dwNumBytes = 8;
if (CryptEncrypt(hKey, NULL, TRUE, 0, byData, &dwNumBytes,
sizeof(byData))) {
// Create a file to store the encrypted data
if ((hFile = CreateFile(_T("C:\\Encrypted.dat"), GENERIC_WRITE,
0, NULL, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL)) !=
INVALID_HANDLE_VALUE) {
// Write the encrypted data out to file
WriteFile(hFile, &byData, dwNumBytes, &dwBytesWritten, NULL);
// Close the file handdle
CloseHandle(hFile);
}
}
// Destroy the session key
CryptDestroyKey(hKey);
}
}
// Destroy the hash object
CryptDestroyHash(hHash);
}
// Release the cryptographic provider context
CryptReleaseContext(hProv, 0);
}

C# Application:
==========
int iNumBytes;
FileStream fsData;
byte[] byData = new byte[16];
byte[] byInitVect = {0,0,0,0,0,0,0,0};

// Open the file containing the encrypted data, read inb the data and
close
the file
fsData = File.Open("C:\\Encrypted.dat", FileMode.Open);
iNumBytes = fsData.Read(byData, 0, byData.Length);
fsData.Close();

// Derive a session key from the password using an MD5 hash
PasswordDeriveBytes SessionKey = new PasswordDeriveBytes("ABCDEF", null,
"MD5", 1);

// Set up an RC2 cryptographic object
RC2CryptoServiceProvider Rc2 = new RC2CryptoServiceProvider();
Rc2.Mode = CipherMode.CBC;
Rc2.KeySize = 128;
Rc2.EffectiveKeySize = 128;
Rc2.BlockSize = 64;

// Decrypt the data
ICryptoTransform decryptor = Rc2.CreateDecryptor(SessionKey.GetBytes(16),
byInitVect);
byte[] myOutputBytes = new byte[decryptor.OutputBlockSize];
iNumBytes = decryptor.TransformBlock(byData, 0, decryptor.InputBlockSize,
myOutputBytes, 0);


Any help would be greatly appreciated.

Thanks,

- Darren -



Doug Barlow said:
Darren,

I haven't tried running your programs yet, but I do have a few
suggestions.

First, you've probably chosen the worst case algorithm for
compatibility --
The old Microsoft CSPs mess with 40-bit RC2 to try to make it stronger,
giving it an 88-byte salt, so you've got to manage the salt values, too.
You might have better luck moving up to a 128-bit RC2 key, which doesn't
default to a special salt value. Take a look at the MSDN documentation
for
CryptDeriveKey for details.

Next, I wouldn't assume the .NET PasswordDeriveBytes class derives the
same
key as the Win32 CryptDeriveKey service. In fact, I'd bet against it.
You
should come up with a common method for deriving a key, and make sure you
use the same key and salt in both applications. You can import a
plain-text
session key into CryptoAPI by following the example shown in
http://support.microsoft.com/default.aspx?scid=kb;en-us;228786.

I hope that helps,

Doug Barlow
The Soft Pedal Shop
http://www.softpedal.net


Hi There,

I have been scanning the newsgroups for a solution to my problem and
have
found that a few others are also experiencing the same problem but none
of
the solutions provided to them seem to work for me.
...
Click to expand...
Click to expand...
Click to expand...
 
D

Darren Bennett

Thanks for your feedback.

I tried specifing (128 << 16) for the 128 bit session key version of the
test code but it still didn't help. My feeling is that there is something
wrong with the padding because Win32 encryption uses "PKCS #1 Type 2 padding"
and .NET encryption does not support this and if I set the PaddingMode.Zeros
in the C# application actually decrypts something but the dat is incorrect.

I have been able to get around this incompatibility problem using Win32
Crypto API via P/Invkoe.

Thanks.

Cheers,

-Darren-
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads

Decrypting in Javascript 8
Windows LLDP Driver Responds With No Data 0
MD5CryptoServiceProvider (ported from CryptHashData) 0
Hash seems incorrect in RC2 0
Dotnetnuke Password Encryption 0
How can I view / open / render / display a pdf file with c code? 0
32/64 bit cc differences 110
How to retrive payment result value after secure paytabs payment done in PayTabs payment gateway in asp.net 0

Members online

No members online now.
Total: 31 (members: 0, guests: 31)
Robots: 611

Forum statistics

Threads
473,769
Messages
2,569,581
Members
45,056
Latest member
GlycogenSupporthealth

Latest Threads

Top