Thanks to this Stackoverflow post for the tip.
After uploading the certificate using the Azure Portal, you need to add an appsetting called WEBSITE_LOAD_CERTIFICATES. The value of the setting is the thumbprint of the certificate. We can also pass multiple thumbprints, delimited by comma or use * to load all available certificates. After this the certificates will be loaded to the personal certificate store of the current user.
The certificate can then be accessed by code
var store = new X509Store(StoreName.My, StoreLocation.CurrentUser);
var certs = store.Certificates.Find(X509FindType.FindByThumbprint, YOUR_THUMBPRINT, false);