Decrypt / Dump contents of CWALLET.SSO (Oracle file based credential store)

When using a file-based credential store with Oracle, credentials ultimately get stored in a wallet file (cwallet.sso)

Very little if any info exists on how to dump the contents of the wallet.  At best, most people leverage the trusty orapki command to get an overview of what’s inside as far as the maps and keys, but actual password information is never divulged.

For example:

$MW_HOME/oracle_common/bin/orapki wallet display -wallet ~/cwallet.sso
Oracle PKI Tool : Version 11.1.1.6.0
Copyright (c) 2004, 2011, Oracle and/or its affiliates. All rights reserved.

Requested Certificates:
User Certificates:
Oracle Secret Store entries:
dip@#3#@cn=odisrv
ODSMMap@#3#@ODSMKey.Wallet
oracle.wsm.security@#3#@enc-csf-key
oracle.wsm.security@#3#@keystore-csf-key
oracle.wsm.security@#3#@sign-csf-key
Trusted Certificates:
Subject:        OU=Class 1 Public Primary Certification Authority,O=VeriSign\, Inc.,C=US
Subject:        OU=Secure Server Certification Authority,O=RSA Data Security\, Inc.,C=US
Subject:        CN=Entrust.net Secure Server Certification Authority,OU=(c) 1999 Entrust.net Limited,OU=www.entrust.net/CPS incorp. by ref. (limits liab.),O=Entrust.net,C=US
Subject:        CN=GTE CyberTrust Global Root,OU=GTE CyberTrust Solutions\, Inc.,O=GTE Corporation,C=US
Subject:        OU=Class 3 Public Primary Certification Authority,O=VeriSign\, Inc.,C=US
Subject:        CN=Entrust.net Secure Server Certification Authority,OU=(c) 2000 Entrust.net Limited,OU=www.entrust.net/SSL_CPS incorp. by ref. (limits liab.),O=Entrust.net
Subject:        OU=Class 2 Public Primary Certification Authority,O=VeriSign\, Inc.,C=US
Subject:        CN=Entrust.net Certification Authority (2048),OU=(c) 1999 Entrust.net Limited,OU=www.entrust.net/CPS_2048 incorp. by ref. (limits liab.),O=Entrust.net

Until now that is :)

Note - wallet and jps-config-dump file must reside in same location for this sample code to function!!  In the example below, choose either the domain wallet, or the bootstrap wallet.

PATH_TO_WALLET="$DOMAIN_HOME/config/fmwconfig/bootstrap/cwallet.sso"
PATH_TO_JPS="$DOMAIN_HOME/config/fmwconfig/bootstrap/jps-config-dump.xml"

or …

PATH_TO_WALLET="$DOMAIN_HOME/config/fmwconfig/cwallet.sso"
PATH_TO_JPS="$DOMAIN_HOME/config/fmwconfig/jps-config-dump.xml"

cat > "${PATH_TO_JPS}" <<EOF
<?xml version="1.0" encoding="UTF-8" standalone='yes'?>
<jpsConfig xmlns="http://xmlns.oracle.com/oracleas/schema/11/jps-config-11_1.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://xmlns.oracle.com/oracleas/schema/11/jps-config-11_1.xsd jps-config-11_1.xsd" schema-major-version="11" schema-minor-version="1">
   <serviceProviders>
      <serviceProvider type="CREDENTIAL_STORE" name="credstoressp" class="oracle.security.jps.internal.credstore.ssp.SspCredentialStoreProvider">
         <description>Credential Store Service Provider</description>
      </serviceProvider>
   </serviceProviders>
   <serviceInstances>
      <serviceInstance provider="credstoressp" name="credstore">
         <property value="file:${PATH_TO_WALLET}" name="location"/>
      </serviceInstance>
   </serviceInstances>
  <jpsContexts default="test">
    <jpsContext name="test">
      <serviceInstanceRef ref="credstore"/>
    </jpsContext>
  </jpsContexts>
</jpsConfig>
EOF

cat > /tmp/DumpWallet.java <<EOF
import java.io.File;

import java.util.Hashtable;

import oracle.security.jps.JpsContext;
import oracle.security.jps.JpsContextFactory;

import oracle.security.jps.service.credstore.Credential;
import oracle.security.jps.service.credstore.CredentialFactory;
import oracle.security.jps.service.credstore.CredentialMap;
import oracle.security.jps.service.credstore.CredentialStore;
import oracle.security.jps.service.credstore.GenericCredential;
import oracle.security.jps.service.credstore.PasswordCredential;

public class DumpWallet
{
  private static final byte[] HEX = new byte[] {
    '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'A', 'B', 'C', 'D', 'E', 'F' }; 
 
  public static void main(String[] args)
  {
    try
    {
      System.setProperty("oracle.security.jps.config", (args.length > 0) ? args[0] : "jps-config.xml");
      // System.setProperty("java.security.debug", "all");
      JpsContextFactory ctxFactory  = JpsContextFactory.getContextFactory();
      JpsContext ctx = ctxFactory.getContext();
      CredentialStore store = ctx.getServiceInstance(CredentialStore.class);
      listAll(store);
    }
    catch (Exception e)
    {
      e.printStackTrace();
    }
  }
 
  private static void listAll(CredentialStore store) throws Exception
  {
    System.out.println("Dumping store contents ...");
    for (String map : store.getMapNames())
    {
      System.out.println("\n" + "### Map: " + map);
      CredentialMap credMap = store.getCredentialMap(map);
      if (credMap != null)
      {
        int i = 1;
        for (String key : credMap.keySet())
        {
          System.out.println(" " + i++ + ". + Key: " + key);
          
          Credential cred = credMap.getCredential(key);
          System.out.println("  class = " + cred.getClass().getName());
          System.out.println("  desc  = " + cred.getDescription());

          if (cred instanceof PasswordCredential)
          {
            PasswordCredential pc = (PasswordCredential)cred;
            System.out.println("  name  = " + pc.getName());
            System.out.println("  pass  = " + new String(pc.getPassword()));
            System.out.println("  expires   = " + pc.getExpiryTime());
          }
          else if (cred instanceof GenericCredential)
          {
            GenericCredential gc = (GenericCredential)cred;
            Object c = gc.getCredential();
            String type = (! c.getClass().isArray())
              ? c.getClass().getName()
              : ("Array of " + c.getClass().getComponentType().getName());

            System.out.println("  type  = " + type);
            if (c instanceof String)
            {
              System.out.println("  cred  = " + c);
            }
            else if ( c instanceof Hashtable)
            {
              Hashtable ht = (Hashtable)c;
              for (Object htkey : ht.keySet())
              {
                Object htVal = ht.get(htkey);
                if (htVal instanceof char[])
                {
                  System.out.println("  cred  = (" + htkey + ", " + new String((char[])htVal) + ")");
                }
                else
                {
                  System.out.println("  cred  = (" + htkey + ", " + htVal + ")");
                }
              }
            }
            else if (c instanceof javax.crypto.spec.SecretKeySpec)
            {
              javax.crypto.spec.SecretKeySpec secret = (javax.crypto.spec.SecretKeySpec) c;
              System.out.println("  algorith  = " + secret.getAlgorithm());
              System.out.println("  format  = " + secret.getFormat());
              System.out.println("  key material as hex = " + bytesAsHex(secret.getEncoded()));
            }
            else if (c instanceof byte[])
            {
              System.out.println("  byte array as hex = " + bytesAsHex((byte[])c));
            }
            System.out.println("  expires   = " + gc.getExpiryTime());
          }
          else
          {
            System.out.println("  toStr = " + cred.toString());
          }
        }
      }
    }
  }

  public static final String bytesAsHex(byte[] bytes)
  {
    StringBuffer sb = new StringBuffer();
    for (int i = 0; i < bytes.length; i++)
    {
      sb.append((char)(HEX[(bytes[i] & 0x00F0) >> 4])).append((char)(HEX[bytes[i] & 0x000F])).append(" ");
    }
    return sb.toString();
  }
}
EOF

CP=/tmp
CP=$CP:$MW_HOME/oracle_common/modules/oracle.jps_11.1.1/jps-api.jar
CP=$CP:$MW_HOME/oracle_common/modules/oracle.jps_11.1.1/jps-common.jar
CP=$CP:$MW_HOME/oracle_common/modules/oracle.jps_11.1.1/jps-internal.jar
CP=$CP:$MW_HOME/oracle_common/modules/oracle.idm_11.1.1/identitystore.jar
CP=$CP:$MW_HOME/oracle_common/modules/oracle.osdt_11.1.1/osdt_xmlsec.jar
CP=$CP:$MW_HOME/oracle_common/modules/oracle.pki_11.1.1/oraclepki.jar

$JAVA_HOME/bin/javac -cp $CP /tmp/DumpWallet.java

$JAVA_HOME/bin/java -cp $CP DumpWallet "${PATH_TO_JPS}"

Sample Output :-

Dumping store contents ...

### Map: oracle.wsm.security
1. + Key: sign-csf-key
  class = oracle.security.jps.internal.credstore.PasswordCredentialImpl
  desc  = signing key alias/password
  name  = orakey
  pass  = welcome1
  expires   = null
2. + Key: enc-csf-key
  class = oracle.security.jps.internal.credstore.PasswordCredentialImpl
  desc  = encryption key alias/password
  name  = orakey
  pass  = welcome1
  expires   = null
3. + Key: keystore-csf-key
  class = oracle.security.jps.internal.credstore.PasswordCredentialImpl
  desc  = keystore access password
  name  = n/a
  pass  = welcome1
  expires   = null
4. + Key: test-appid-key
  class = oracle.security.jps.internal.credstore.PasswordCredentialImpl
  desc  = null
  name  = weblogic
  pass  = welcome1
  expires   = null

### Map: IDCCS
1. + Key: ldap:1340771431089
  class = oracle.security.jps.internal.credstore.PasswordCredentialImpl
  desc  = null
  name  = ldap:1340771431089
  pass  = MjcxN0M2ODREOEQ0RjZERg==
  expires   = null
2. + Key: db:1340771431083
  class = oracle.security.jps.internal.credstore.PasswordCredentialImpl
  desc  = null
  name  = db:1340771431083
  pass  = MTlDQTE1N0EzQzE3REY1OA==
  expires   = null
3. + Key: hash:1340771431089
  class = oracle.security.jps.internal.credstore.PasswordCredentialImpl
  desc  = null
  name  = hash:1340771431089
  pass  = MTk1MTk1QkQ4OUE2QzJBNw==
  expires   = null
4. + Key: proxy:1340771431089
  class = oracle.security.jps.internal.credstore.PasswordCredentialImpl
  desc  = null
  name  = proxy:1340771431089
  pass  = M0IwMUZBQjNGQTUxNzk0OA==
  expires   = null

In a WebLogic domain, you will find that there is also a cwallet.sso found in $DOMAIN_HOME/config/fmwconfig/bootstrap.

Dumping store contents ...

### Map: fks
1. + Key: master.key.0
  class = oracle.security.jps.internal.credstore.GenericCredentialImpl
  desc  = null
  type  = javax.crypto.spec.SecretKeySpec
  algorith  = AES
  format  = RAW
  key material as hex = CB 45 4F B0 F8 26 FF 04 31 9F 48 DD 43 42 69 C7
  expires   = null
2. + Key: current.key
  class = oracle.security.jps.internal.credstore.GenericCredentialImpl
  desc  = null
  type  = java.lang.String
  cred  = master.key.0
  expires   = null

### Map: IntegrityChecker
1. + Key: kss
  class = oracle.security.jps.internal.credstore.GenericCredentialImpl
  desc  = null
  type  = Array of byte
  byte array as hex = AB 18 CD 76 6C 39 FE 46 A0 6D 1C F0 BC 8D 97 3A D1 64 BC 80 2C 33 64 8E AE C9 B1 63 88 BE 23 7C 37 2F 63 9D 55 2B 5E 8F 1E 08 0A 73 F1 A8 15 83 8F 24 3D 19 B8 79 6E 75 B2 1C 7F DB 72 FC AE BA 72 A3 62 62 27 29 EE DE
  expires   = null

If you ever decide to reassociate the credential store with LDAP (e.g. using the reassociateSecurityStore command), you will find that the bootstrap cwallet.sso will contain credentials to access the LDAP store.  Here is what my domain wallet files look like when using an LDAP credential store :-

$DOMAIN_HOME/config/fmwconfig/cwallet.sso :

Dumping store contents ...

### Map: dip
1. + Key: cn=odisrv
  class = oracle.security.jps.internal.credstore.PasswordCredentialImpl
  desc  = DIP Password
  name  = cn=odisrv,cn=Registered Instances,cn=Directory Integration Platform,cn=Products,cn=OracleContext
  pass  = YNDMSU1wP1WergcX
  expires   = null

### Map: ODSMMap
1. + Key: ODSMKey.Wallet
  class = oracle.security.jps.internal.credstore.PasswordCredentialImpl
  desc  = ODSM Key store password
  name  = ODSM
  pass  = 0000000000
  expires   = null

$DOMAIN_HOME/config/fmwconfig/bootstrap/cwallet.sso :

Dumping store contents ...

### Map: BOOTSTRAP_JPS
1. + Key: bootstrap_q6ShJcm89vO8N2oVoSFqTLuW6Sg=
  class = oracle.security.jps.internal.credstore.PasswordCredentialImpl
  desc  = bootstrap user name and password
  name  = cn=orcladmin
  pass  = welcome1
  expires   = null

### Map: fks
1. + Key: master.key.0
  class = oracle.security.jps.internal.credstore.GenericCredentialImpl
  desc  = null
  type  = javax.crypto.spec.SecretKeySpec
  algorith  = AES
  format  = RAW
  key material as hex = 1C B5 89 2A 45 F2 BA A0 E5 C1 A8 F6 DE 6E FC 5A
  expires   = null
2. + Key: current.key
  class = oracle.security.jps.internal.credstore.GenericCredentialImpl
  desc  = null
  type  = java.lang.String
  cred  = master.key.0
  expires   = null

### Map: IntegrityChecker
1. + Key: kss
  class = oracle.security.jps.internal.credstore.GenericCredentialImpl
  desc  = null
  type  = Array of byte
  byte array as hex = FD 01 1E 54 D5 84 3B 9D AF DA 62 62 22 BB 7E A9 0C DB 08 A3 D9 71 9F A6 03 96 7F DF 29 69 37 55 60 5D 0E 32 EE 3A D0 D6 F2 A9 FD 58 DB 82 87 A0 98 D2 78 6A 47 48 E9 6B 86 3E 68 77 BC 17 01 B6 A0 BD 29 A2 3B E7 B7 73
  expires   = null

 

Have fun!

JDeveloper - RIDC JAX/WS SAML Connections to UCM / Content Server

I posted a few years back some really useful information on configuring the infrastructure of the JDeveloper 11g embedded WebLogic server to allow consumption of a JAX/WS Web Service protected by a WSM policy:

http://todayguesswhat.blogspot.com.au/2010/09/jdeveloper-11gr1-integrated-weblogic.html

Today I will extend this information with some useful tips for connecting an application that is deployed to the JDeveloper embedded WebLogic server to connect to a back-end Oracle Content Server (aka Oracle UCM; aka Oracle WebCenter Content Server) using JAX/WS protocol.

Firstly, some background on the configuration of the UCM Server (11.1.1.6.0) and its associated domain:

The UCM Server is deployed to a WebLogic Domain (which I now will refer to as the UCM Domain) configured with WSM policy manager; Such that, the oracle.wsmpm_template_11.1.1.jar template has been applied and subsequently configured to connect to the MDS schema installed by the RCU.

The UCM Domain has been configured with an OID Authentication Provider.

A keystore has been created for the UCM Domain and populated with key-certificate pair signed by the demo certificate authority (CA) that ships with WebLogic.  Note – don’t use this CA in production!

oracle.wsm.security credential store entries have been created for the UCM Domain with the credentials / aliases required to access the keystore contents from above.

GPA Policy has also been configured on the UCM Domain for “ws-service” using message protection service policy oracle/wss11_saml_or_username_token_with_message_protection_service_policy which supports username/password connections and saml connections.

Below are some sample script calls that you can leverage to achieve the above UCM server configuration (assuming Linux):

cd $DOMAIN_HOME/config/fmwconfig

# Create service key-certificate pair signed by the demo CA cert "CertGenCA" with key password welcome1
java utils.CertGen -certfile MyPublicCert -keyfile MyPrivateKey -keyfilepass welcome1 -cn "`hostname -f`"

# Create and populate new service keystore default-keystore.jks with service key-certificate pair from above under alias orakey with keystore password welcome1, and alias password welcome1
java utils.ImportPrivateKey -keystore default-keystore.jks -storepass welcome1 -certfile MyPublicCert.der -keyfile MyPrivateKey.der -keyfilepass welcome1 -alias orakey -keypass welcome1

# Now add the root CA to the service keystore under the alias name CA
keytool -importcert -noprompt -alias CA -file $WL_HOME/server/lib/CertGenCA.der -keystore default-keystore.jks -storepass welcome1

$MW_HOME/oracle_common/common/bin/wlst.sh

wls_username="weblogic"
wls_password="welcome1"
wls_url="t3://localhost:7001"
connect(wls_username, wls_password, wls_url)

createCred(map="oracle.wsm.security", key="keystore-csf-key", user="n/a", password="welcome1", desc="keystore access password")
createCred(map="oracle.wsm.security", key="sign-csf-key", user="orakey", password="welcome1", desc="signing key alias/password")
createCred(map="oracle.wsm.security", key="enc-csf-key", user="orakey", password="welcome1", desc="encryption key alias/password")

serverConfig()

domain_name=cmo.getName()
policy_set_name=domain_name+"-ws-service"
policy_type="ws-service"
resource='Domain("' + domain_name + '")'
policy="oracle/wss11_saml_or_username_token_with_message_protection_service_policy"

beginRepositorySession()
createPolicySet(policy_set_name, policy_type, resource)
attachPolicySetPolicy(policy)
validatePolicySet()
commitRepositorySession()
disconnect()
exit()

Note above, we have leveraged a Message Protection service policy.  If we were to switch to a non-Message Protection service policy such as oracle/wss_saml_or_username_token_service_policy life would be much simpler.  However you need to be extremely careful with deploying such a policy in your enterprise.  You need to understand the consequences, as you may be creating yourself an easily exploitable security hole!!!

The policy oracle/wss_saml_or_username_token_service_policy does NOT do signing/encryption, certificates are NOT used!.
Thus the client does not need a keystore configured.  Also, setting up trusted client DN for the issuer will have no affect.
This profile is in no way shape or form secure and should only be used on a closed internal network where no rogue client can exist.
A rogue client can simply leverage the client policy oracle/wss10_saml_token_client_policy and set BindingProvider.USERNAME_PROPERTY to any user they like and make a connection to the service. 

I exploit this backdoor all the time here at Oracle with our fusion test / WAR room environments.  Effectively, you get an EVP-level visibility bug filed against you (which 98% of time is not a bug – at least in my code ;) ) – and you are forced to troubleshoot with a read-only account (if you are lucky) or some minimal obscure log messages.  Using this trick, I just check the UCM login service to see what policy is applied, and if no message protection policy is defined, I go get myself an admin connection and find out what going on before 92ing the bug!

Anyway, back to the topic at hand.  We are using a Message Protection policy, so the client must have a valid keystore and key-certificate pair in use, and the client’s public certificate must be present in the server’s keystore.  The simplest mechanism in a non-security hardened test environment to achieve this requirement is to simply copy/clone the server’s keystore file to the client, and create identical credential store entries for the oracle.wsm.security properties.  In security hardened environments though, each client domain would have a unique signing/encryption keypair, and the client's associated public certificate must be manually imported in to the server domain's keystore.

Let’s proceed with the JDeveloper embedded WebLogic client steps based on simple cloning of the server’s keystore! If you have not already created the default domain in JDeveloper, do so now, and fire up the server!  I’m running Oracle JDeveloper 11g Release 2 (11.1.2.1.0) on Windows XP.

1. From the JDeveloper View menu, choose Application Server Navigator
2. Right click on Application Servers / IntegratedWebLogicServer, select "Create Default Domain..."
2. Right click on Application Servers / IntegratedWebLogicServer, select "Start Server Instance"

Step 1) Copy from the UCM server its default-keystore.jks file.

/u01/app/oracle/product/Middleware/user_projects/domains/base_domain/config/fmwconfig/default-keystore.jks

Copy this file to the %DOMAIN_HOME%\config\fmwconfig directory of your jdeveloper

e.g. from Command Prompt:

SET DOMAIN_HOME=C:\JDeveloper\system11.1.2.1.38.60.81\DefaultDomain
%DOMAIN_HOME%\bin\setDomainEnv.cmd
cd %DOMAIN_HOME%\config\fmwconfig
## ftp binary get from UCM server the following file
/u01/app/oracle/product/Middleware/user_projects/domains/base_domain/config/fmwconfig/default-keystore.jks

Step 2) Create credential store entries to access keys/certificates from keystore file above (make sure your domain is running!)

%MW_HOME%\oracle_common\common\bin\wlst
connect('weblogic','welcome1','t3://127.0.0.1:7101')
createCred(map="oracle.wsm.security", key="keystore-csf-key", user="n/a", password="welcome1")
createCred(map="oracle.wsm.security", key="sign-csf-key", user="orakey", password="welcome1")
createCred(map="oracle.wsm.security", key="enc-csf-key", user="orakey", password="welcome1")
exit()

Step 3) Create GPA ws-client policy

%MW_HOME%\oracle_common\common\bin\wlst
connect('weblogic','welcome1','t3://127.0.0.1:7101')

serverConfig()

domain_name=cmo.getName()
policy_set_name=domain_name+"-ws-client"
policy_type="ws-client"
resource='Domain("' + domain_name + '")'
policy="oracle/wss11_saml_token_with_message_protection_client_policy"

beginRepositorySession()
createPolicySet(policy_set_name, policy_type, resource)
attachPolicySetPolicy(policy)
validatePolicySet()
commitRepositorySession()
disconnect()
exit()

Step 4) Configure JDeveloper Embedded WebLogic Domain to use OID Authentication Provider and OID server leveraged by the UCM server.

%MW_HOME%\oracle_common\common\bin\wlst
connect('weblogic','welcome1','t3://127.0.0.1:7101')

edit()
startEdit()
securityRealm = cmo.getSecurityConfiguration().getDefaultRealm();
print("Realm: " + securityRealm.getName())

oid_provider_name="OIDAuthenticator"
oid_host="xxx.us.oracle.com"
oid_port=3060
oid_user="cn=orcladmin"
oid_user_pwd="welcome1"
oid_userbase="cn=Users,dc=us,dc=oracle,dc=com"
oid_groupbase="cn=Groups,dc=us,dc=oracle,dc=com"

oidAP = securityRealm.createAuthenticationProvider(oid_provider_name,"weblogic.security.providers.authentication.OracleInternetDirectoryAuthenticator")
oidAP.setHost(oid_host)
oidAP.setPort(oid_port)
oidAP.setPrincipal(oid_user)
oidAP.setCredential(oid_user_pwd)
oidAP.setUserBaseDN(oid_userbase)
oidAP.setAllUsersFilter("(&(uid=*)(objectclass=person))")
oidAP.setUserNameAttribute("uid")
oidAP.setUseRetrievedUserNameAsPrincipal(true)
oidAP.setGroupBaseDN(oid_groupbase)
oidAP.setControlFlag("SUFFICIENT")

print("Provider order (existing) : ")
print securityRealm.getAuthenticationProviders()
daAP = securityRealm.lookupAuthenticationProvider("DefaultAuthenticator")
diaAP = securityRealm.lookupAuthenticationProvider("DefaultIdentityAsserter")
securityRealm.setAuthenticationProviders([oidAP,daAP,diaAP])
print("Provider order (updated) : ")
print securityRealm.getAuthenticationProviders()

print("Default Authenticator Control Flag (existing) : " + daAP.getControlFlag())
daAP.setControlFlag("SUFFICIENT")
print("Default Authenticator Control Flag (updated) : " + daAP.getControlFlag())

save()
activate(block="true")
disconnect()
exit()

Step 5) Optional - update WLS admins ...

%MW_HOME%\oracle_common\common\bin\wlst
connect('weblogic','welcome1','t3://127.0.0.1:7101')

securityRealm = cmo.getSecurityConfiguration().getDefaultRealm()
roleMapper = securityRealm.lookupRoleMapper("XACMLRoleMapper")
existing = roleMapper.getRoleExpression(None,"Admin")
print("WLS Admins (existing) : " +  existing)

ecm_domain_admin_oid_group_cn="ECM Domain Administrators"

roleMapper.setRoleExpression(None,"Admin",existing + "|Grp(" + ecm_domain_admin_oid_group_cn + ")")
print("WLS Admins (updated) : " +  roleMapper.getRoleExpression(None,"Admin"))

disconnect()
exit()

This should take care of all the infrastructure steps required.  Now it is just a matter of creating a little web application in JDeveloper that leverages RIDC to connect to the UCM Server using JAX/WS protocol and GPA client policy.

File > New > General > Applications > Custom Application
Application Name: RIDCJAXWSTest
Project Name: web
Product Features: JSP and Servlets

Right click on web project > New > Web Tier > Servlets > HTTP Servlet
Web Application Version: Servlet 2.5\JSP 2.1
Servlet class: RIDCTestServlet
Servlet package: web
Mapping details:
Name: RIDCTestServlet
URL Pattern /app

Right click on web project > Project Properties > Libraries and Classpath
Add and select a new user library with classpath referencing oracle.ucm.ridc-11.1.1.jar

Servlet and XML Code

web/RIDCTestServlet.java :-

package web;

import java.io.IOException;
import java.io.PrintWriter;

import java.util.Map;

import javax.servlet.*;
import javax.servlet.http.*;

import oracle.stellent.ridc.IdcClient;
import oracle.stellent.ridc.IdcClientException;
import oracle.stellent.ridc.IdcClientManager;
import oracle.stellent.ridc.IdcContext;

import oracle.stellent.ridc.model.DataBinder;
import oracle.stellent.ridc.model.DataObject;

import oracle.stellent.ridc.protocol.ServiceResponse;

public class RIDCTestServlet extends HttpServlet
{
  private static final String url = "http://ucmserver.com:16200/idcnativews";
   
  private IdcClient m_idcClient = null;

  public void init(ServletConfig config) throws ServletException
  {
    super.init(config);
    IdcClientManager clientManager = new IdcClientManager();
    try
    {
      m_idcClient = clientManager.createClient(url);
    }
    catch (IdcClientException e)
    {
      e.printStackTrace();
      throw new RuntimeException(e);
    }

    // enable verbose ridc logging to console
    oracle.stellent.ridc.common.log.LogFactory.setLogProvider(
      new oracle.stellent.ridc.common.log.simple.SimpleLogProvider());
  }

  protected void doGet(HttpServletRequest request, HttpServletResponse response)
    throws ServletException, IOException
  {
    doPost(request, response);
  }

  protected void doPost(HttpServletRequest request, HttpServletResponse response)
    throws ServletException, IOException
  {
    PrintWriter out = response.getWriter();
    out.print("<html><head><title>RIDC Test Servlet</title></head><body><pre>");
   
    try
    {
   
      out.print("RIDC Version: " + m_idcClient.getVersion() + "<br/><hr/>");
     
      String username = request.getRemoteUser();
      out.print("request.getRemoteUser() = " + username + "<br/><hr/>");
     
      // we are using SAML
      IdcContext userContext = userContext = new IdcContext(username);

      // get the binder
      DataBinder binder = m_idcClient.createBinder();
 
      // populate the binder with the parameters
      binder.putLocal("IdcService", "PING_SERVER");

      out.print("Sending PING_SERVER request ...<br/>");

      // execute the request
      ServiceResponse resp = m_idcClient.sendRequest(userContext, binder);
 
      // get the binder - get a binder closes the response automatically
      DataBinder responseBinder = resp.getResponseAsBinder();
     
      out.print("<hr/>PING_SERVER response ...<br/>");

      DataObject localData = responseBinder.getLocalData();
      for (Map.Entry<String, String> entry : localData.entrySet())
      {
        out.print(entry.getKey() + "=" + entry.getValue() + "<br/>");
      }
     
      m_idcClient.logout(userContext); // only new versions of RIDC support this
    }
    catch (Exception e)
    {
      out.print("Exception: <br/><hr/>");
      e.printStackTrace(out);
    }
    finally
    {
      out.print("</pre></body></html>");
    }
  }
}

WEB-INF/web.xml :-

<?xml version = '1.0' encoding = 'UTF-8'?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"
         version="2.5">
  <servlet>
    <servlet-name>RIDCTestServlet</servlet-name>
    <servlet-class>web.RIDCTestServlet</servlet-class>
  </servlet>
  <servlet-mapping>
    <servlet-name>RIDCTestServlet</servlet-name>
    <url-pattern>/app</url-pattern>
  </servlet-mapping>

  <security-constraint>
    <web-resource-collection>
      <web-resource-name>Constraint-0</web-resource-name>
      <url-pattern>/*</url-pattern>
    </web-resource-collection>
    <auth-constraint>
      <role-name>valid-users</role-name>
    </auth-constraint>
  </security-constraint>

  <login-config>
    <auth-method>BASIC</auth-method>
    <realm-name>myrealm</realm-name>
  </login-config>

  <security-role>
    <role-name>valid-users</role-name>
  </security-role>
</web-app>

 

WEB-INF/weblogic.xml :-

<?xml version="1.0" encoding="UTF-8"?>
<wls:weblogic-web-app xmlns:wls="http://xmlns.oracle.com/weblogic/weblogic-web-app" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-

app_2_5.xsd http://xmlns.oracle.com/weblogic/weblogic-web-app http://xmlns.oracle.com/weblogic/weblogic-web-app/1.0/weblogic-web-app.xsd">

  <wls:security-role-assignment>
    <wls:role-name>valid-users</wls:role-name>
    <wls:principal-name>users</wls:principal-name>
  </wls:security-role-assignment>

  <!--
  the users group is automatically made available to the security realm;
  this group represents the set of all authenticated users.
  Any user that successfully authenticates itself is a member of the users group.
  -->
</wls:weblogic-web-app>

 

Sample Output:  (after authenticating as user ecmadmin)

RIDC Version: 11.1.1.6.3.7848
request.getRemoteUser() = ecmadmin
Sending PING_SERVER request ...
PING_SERVER response ...
UserDateFormat=iso8601
UserTimeZone=UTC
ClientEncoding=UTF-8
IdcService=PING_SERVER
dUser=ecmadmin
refreshSubjects=
blDateFormat=yyyy-MM-dd HH:mm:ssZ!tUTC!mAM,PM
ECID-Context=1.8987b47a9955d4f6:328ee95f:1382badfe3d:-8000-0000000000000039;kXjE
refreshMonikers=
changedSubjects=
refreshSubMonikers=
blFieldTypes=xForceFolderSecurity text,xPartitionId text,dInDate date,xReadOnly text,xPartnerLevel bigtext,dMessage message,xInhibitUpdate text,xPartnerProgram bigtext,dCreateDate date,xWebFlag text,xHidden text,xPartnerType bigtext,dReleaseDate date,StatusMessage message,xCollectionID int,xStorageRule text,dOutDate date,xExternalDataSet bigtext,xComments memo,xIdcProfile text
NoHttpHeaders=0
changedMonikers=
idcToken=
StatusMessage=You are logged in as 'ecmadmin'.
hasUserAccessChanged=1
IsJava=1
localizedForResponse=1

Spaces wrongly added in OCR text of PDF generated by Abbyy FineReader - Solution

In my search for the holy grail of document scanning and management software, I’ve been trialing Abbyy FineReader 11 and Adobe Acrobat X to see if either offer any benefits over Lucion FileCenter on the scanning front.  FileCenter is a nice piece of software for managing of scans, but it falls short in my opinion on the OCR front. 

What I’m hoping to find in a product is something that offers a high level of OCR accuracy, and the ability to easily correct mistakes with a quick and simple interface (preferably all controlled by keyboard commands for improved processing time).

I’m after a product that could learn / be trained to detect document types and only OCR the data of actual interest.

I’m after a product that easily and automatically recognize document paper size and allows simple and effective trim/deskew/cleanup etc as necessary.

Anyway, these requirements culminated in my testing of FineReader and Acrobat X.

Acrobat X has the potential to be a useful piece of software, but it surprisingly performed quite average on the OCR front with the documents that I threw at it.  The OCR results were average – and that is being kind.  What astonished me however, was the product’s inability to provide a simple mechanism to correct the OCR mistakes.  I had to google search for a way to achieve it; and if anything, the results returned were more a workaround/hack than out-of-the-box intended functionality.  What it did do nicely was the simple one click scan to PDF.  It was just a pity the OCR let it down.

Next up was FineReader 11.  This software appeared to excel on the OCR front, and also allow a simple verification mechanism to correct issues detected.  It however was not without its warts.   Features that could turn this very good software in to excellent software:

1. The ability to perform all document verification/correction steps completely using keyboard with absolutely no mouse interaction required.
2. The ability to save scanning profiles for specific document types.  For example what type of fonts to expect, what paper size, what spelling mistakes/words to ignore.
3. The ability for spell checker / verifier to ignore single letters that appear as part of a word grouping.  For example,  John Smith is fine.  But it sees an issue with the “J” in J Smith.  I should be able to add “J Smith” to my dictionary ignore list.
4. The ability to completely skip OCR on specific pages of a document without silly workarounds.  Currently, you have to add a scan region on the page, and read the page.
5. The ability to better recognize font families and the like.

One major flaw I found in the software, fortunately appears to have a cure. I was hoping others would have run in to it and provided a solution in some type of forum, but alas that was not the case.  Abbyy – if you are reading this post by any chance, I would strongly suggest you consider making a public support forum for people to discuss your software and offer tips etc.  It should definitely lead to more sales if people can find solutions to problems they encounter with your software.

Anyway, the issue I ran in to was the following:

When converting scan to PDF, words had invalid spacing/padding added not present in either the OCR text in Abbyy, or for that matter, in other output options such as HTML etc.

The cause:

The Windows machine likely does not have the matching font installed, and Abbyy likely not configured to leverage the font.

The solution:

First attempt to identify the font leveraged by the document.  If only a scan is available, extract some words containing sufficient sample characters and head on over to http://www.myfonts.com/WhatTheFont/ and attempt to determine the font.

If you can get hold of an electronic PDF of the same file (or similar from same provider), you should be able to go to Acrobat > File > Properties dialog > Fonts tab.  If an electronic version of the document was available, within Acrobat you will likely see that the font has been embedded, but only
a portion of the font was embedded (such that the characters leveraged by the document, and nothing else).

Using tools such as FontForge and mupdf-1.0-tools-windows.zip, you can potentially extract the embedded fonts subsets, and try and convert these to TTF.  But it is likely the TTF won't be sufficient.  Your best bet is to try and find the original TTF on the net.  You may have to purchase it.

One other issue, is that even if you can find the TTF, chances are licensing flags on the TTF will prevent it from being embedded in a PDF document.
To circumvent this, you can leverage ttfpatch and set the fsType flag to 0, meaning: Installable embedding allowed, fonts may be embedded in documents and permanently installed on the remote system.

Install the font then in the <windows>\fonts directory, and open Abbyy.
From Tools > Options > Read tab > Fonts > ensure the font is selected (and thus available for OCR).

If saving the file as PDF/A (which embeds fonts), and an error message appears stating the font is restricted from being embeeded, first ensure the fsType flag has been reset to 0, and also try and clear out Abbyy font cache.  Delete directory
C:\Documents and Settings\All Users\Application Data\ABBYY\FineReader\11.00\FontCache

Also, it is worthwhile installing Microsoft Typography Font Properties Extension
http://www.microsoft.com/typography/FreeToolsOverview.mspx
This will allow you to right click > properties on a TTF file, and see the Embedding restrictions on the file.

Good luck!