tag:blogger.com,1999:blog-29594981052391836422024-03-27T19:23:30.600+10:00Today, guess what ...Techniques and thoughts worth documenting.Matt (Brisbane/Australia)http://www.blogger.com/profile/16649081553551301779noreply@blogger.comBlogger62125tag:blogger.com,1999:blog-2959498105239183642.post-88921184492022984792021-01-28T19:57:00.004+10:002021-01-28T20:13:28.823+10:00Manually verifying RSA SHA Signature in Java using Cipher<div style="text-align: left;"><span style="font-family: courier;">The post below may be helpful in understanding how a signature is generated and verified.</span></div><h2 style="text-align: left;"><span style="font-family: courier;">Sample code:</span></h2><p><div style="text-align: left; line-height: .4em;"><span style="font-family: courier; font-size: x-small;">import java.security.KeyPair;</span></p><p><span style="font-family: courier; font-size: x-small;">import java.security.KeyPairGenerator;</span></p><p><span style="font-family: courier; font-size: x-small;">import java.security.MessageDigest;</span></p><p><span style="font-family: courier; font-size: x-small;">import java.security.PrivateKey;</span></p><p><span style="font-family: courier; font-size: x-small;">import java.security.PublicKey;</span></p><p><span style="font-family: courier; font-size: x-small;">import java.security.Signature;</span></p><p><span style="font-family: courier; font-size: x-small;"><br /></span></p><p><span style="font-family: courier; font-size: x-small;">import java.util.Arrays;</span></p><p><span style="font-family: courier; font-size: x-small;"><br /></span></p><p><span style="font-family: courier; font-size: x-small;">import javax.crypto.Cipher;</span></p><p><span style="font-family: courier; font-size: x-small;"><br /></span></p><p><span style="font-family: courier; font-size: x-small;">import sun.security.util.DerInputStream;</span></p><p><span style="font-family: courier; font-size: x-small;">import sun.security.util.DerValue;</span></p><p><span style="font-family: courier; font-size: x-small;"><br /></span></p><p><span style="font-family: courier; font-size: x-small;">public class RSASignatureVerification</span></p><p><span style="font-family: courier; font-size: x-small;">{</span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>public static void main(String[] args) throws Exception</span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{</span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>KeyPairGenerator generator = KeyPairGenerator.getInstance("RSA");</span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>generator.initialize(2048);</span></p><p><span style="font-family: courier; font-size: x-small;"><br /></span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>KeyPair keyPair = generator.generateKeyPair();</span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>PrivateKey privateKey = keyPair.getPrivate();</span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>PublicKey publicKey = keyPair.getPublic();</span></p><p><span style="font-family: courier; font-size: x-small;"><br /></span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>String data = "hello mshannon";</span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>byte[] dataBytes = data.getBytes("UTF8");</span></p><p><span style="font-family: courier; font-size: x-small;"><br /></span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>Signature signer = Signature.getInstance("SHA512withRSA");</span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>signer.initSign(privateKey);</span></p><p><span style="font-family: courier; font-size: x-small;"><br /></span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>signer.update(dataBytes);</span></p><p><span style="font-family: courier; font-size: x-small;"><br /></span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>byte[] signature = signer.sign(); // signature bytes of the signing operation's result.</span></p><p><span style="font-family: courier; font-size: x-small;"><br /></span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>Signature verifier = Signature.getInstance("SHA512withRSA");</span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>verifier.initVerify(publicKey);</span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>verifier.update(dataBytes);</span></p><p><span style="font-family: courier; font-size: x-small;"><br /></span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>boolean verified = verifier.verify(signature);</span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>if (verified)</span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{</span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>System.out.println("Signature verified!");</span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>}</span></p><p><span style="font-family: courier; font-size: x-small;"><br /></span></p><p><span style="font-family: courier; font-size: x-small;">/*</span></p><p><span style="font-family: courier; font-size: x-small;"> The statement that describes signing to be equivalent to RSA encrypting the</span></p><p><span style="font-family: courier; font-size: x-small;"> hash of the message using the private key is a greatly simplified view</span></p><p><span style="font-family: courier; font-size: x-small;"> The decrypted signatures bytes likely convey a structure (ASN.1) encoded</span></p><p><span style="font-family: courier; font-size: x-small;"> using DER with the hash just one component of the structure.</span></p><p><span style="font-family: courier; font-size: x-small;">*/</span></p><p><span style="font-family: courier; font-size: x-small;"><br /></span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>// lets try decrypt signature and see what is in it ...</span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>Cipher cipher = Cipher.getInstance("RSA");</span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>cipher.init(Cipher.DECRYPT_MODE, publicKey);</span></p><p><span style="font-family: courier; font-size: x-small;"><br /></span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>byte[] decryptedSignatureBytes = cipher.doFinal(signature);</span></p><p><span style="font-family: courier; font-size: x-small;"><br /></span></p><p><span style="font-family: courier; font-size: x-small;">/*</span></p><p><span style="font-family: courier; font-size: x-small;"> sample value of decrypted signature which was 83 bytes long</span></p><p><span style="font-family: courier; font-size: x-small;"><br /></span></p><p><span style="font-family: courier; font-size: x-small;"> 30 51 30 0D 06 09 60 86 48 01 65 03 04 02 03 05</span></p><p><span style="font-family: courier; font-size: x-small;"> 00 04 40 51 00 41 75 CA 3B 2B 6B C0 0A 3F 99 E3</span></p><p><span style="font-family: courier; font-size: x-small;"> 6B 7A 01 DC F2 9B 36 E6 0D D4 31 89 53 A3 D9 80</span></p><p><span style="font-family: courier; font-size: x-small;"> 6D AE DD 45 7E 55 45 01 FC C8 73 D2 DD 8D E5 B9</span></p><p><span style="font-family: courier; font-size: x-small;"> E0 71 57 13 41 D0 CD FF CA 58 01 03 A3 DD 95 A1</span></p><p><span style="font-family: courier; font-size: x-small;"> C1 EE C8</span></p><p><span style="font-family: courier; font-size: x-small;"><br /></span></p><p><span style="font-family: courier; font-size: x-small;"> Taking above sample bytes ...</span></p><p><span style="font-family: courier; font-size: x-small;"> 0x30 means A SEQUENCE - which contains an ordered field of one or more types.</span></p><p><span style="font-family: courier; font-size: x-small;"> It is encoded into a TLV triplet that begins with a Tag byte of 0x30.</span></p><p><span style="font-family: courier; font-size: x-small;"> DER uses T,L,V (tag bytes, length bytes, value bytes) format</span></p><p><span style="font-family: courier; font-size: x-small;"><br /></span></p><p><span style="font-family: courier; font-size: x-small;"> 0x51 is the length = 81 decimal (13 bytes)</span></p><p><span style="font-family: courier; font-size: x-small;"><br /></span></p><p><span style="font-family: courier; font-size: x-small;"> the 0x30 (48 decimal) that follows begins a second sequence</span></p><p><span style="font-family: courier; font-size: x-small;"><br /></span></p><p><span style="font-family: courier; font-size: x-small;"> https://tools.ietf.org/html/rfc3447#page-43</span></p><p><span style="font-family: courier; font-size: x-small;"> the DER encoding T of the DigestInfo value is equal to the following for SHA-512</span></p><p><span style="font-family: courier; font-size: x-small;"> 0D 06 09 60 86 48 01 65 03 04 02 03 05 00 04 40 || H</span></p><p><span style="font-family: courier; font-size: x-small;"> where || is concatenation and H is the hash value.</span></p><p><span style="font-family: courier; font-size: x-small;"><br /></span></p><p><span style="font-family: courier; font-size: x-small;"> 0x0D is the length = 13 decimal (13 bytes)</span></p><p><span style="font-family: courier; font-size: x-small;"><br /></span></p><p><span style="font-family: courier; font-size: x-small;"> 0x06 means an OBJECT_ID tag</span></p><p><span style="font-family: courier; font-size: x-small;"> 0x09 means the object id is 9 bytes ...</span></p><p><span style="font-family: courier; font-size: x-small;"><br /></span></p><p><span style="font-family: courier; font-size: x-small;"> https://docs.microsoft.com/en-au/windows/win32/seccertenroll/about-object-identifier?redirectedfrom=MSDN</span></p><p><span style="font-family: courier; font-size: x-small;"><br /></span></p><p><span style="font-family: courier; font-size: x-small;"> taking 2.16.840.1.101.3.4.2.3 - (object id for SHA512 Hash Algorithm)</span></p><p><span style="font-family: courier; font-size: x-small;"><br /></span></p><p><span style="font-family: courier; font-size: x-small;"> The first two nodes of the OID are encoded onto a single byte.</span></p><p><span style="font-family: courier; font-size: x-small;"> The first node is multiplied by the decimal 40 and the result is added to the value of the second node</span></p><p><span style="font-family: courier; font-size: x-small;"> 2 * 40 + 16 = 96 decimal = 60 hex</span></p><p><span style="font-family: courier; font-size: x-small;"> Node values less than or equal to 127 are encoded on one byte.</span></p><p><span style="font-family: courier; font-size: x-small;"> 1 101 3 4 2 3 corresponds to in hex 01 65 03 04 02 03</span></p><p><span style="font-family: courier; font-size: x-small;"> Node values greater than or equal to 128 are encoded on multiple bytes.</span></p><p><span style="font-family: courier; font-size: x-small;"> Bit 7 of the leftmost byte is set to one. Bits 0 through 6 of each byte contains the encoded value.</span></p><p><span style="font-family: courier; font-size: x-small;"> 840 decimal = 348 hex</span></p><p><span style="font-family: courier; font-size: x-small;"> -> 0000 0011 0100 1000</span></p><p><span style="font-family: courier; font-size: x-small;"> set bit 7 of the left most byte to 1, ignore bit 7 of the right most byte,</span></p><p><span style="font-family: courier; font-size: x-small;"> shifting right nibble of leftmost byte to the left by 1 bit</span></p><p><span style="font-family: courier; font-size: x-small;"> -> 1000 0110 X100 1000 in hex 86 48</span></p><p><span style="font-family: courier; font-size: x-small;"><br /></span></p><p><span style="font-family: courier; font-size: x-small;"> 05 00 ; NULL (0 Bytes)</span></p><p><span style="font-family: courier; font-size: x-small;"><br /></span></p><p><span style="font-family: courier; font-size: x-small;"> 04 40 ; OCTET STRING (0x40 Bytes = 64 bytes</span></p><p><span style="font-family: courier; font-size: x-small;"> SHA512 produces a 512-bit (64-byte) hash value</span></p><p><span style="font-family: courier; font-size: x-small;"><br /></span></p><p><span style="font-family: courier; font-size: x-small;"> 51 00 41 ... C1 EE C8 is the 64 byte hash value</span></p><p><span style="font-family: courier; font-size: x-small;">*/</span></p><p><span style="font-family: courier; font-size: x-small;"><br /></span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>// parse DER encoded data</span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>DerInputStream derReader = new DerInputStream(decryptedSignatureBytes);</span></p><p><span style="font-family: courier; font-size: x-small;"><br /></span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>byte[] hashValueFromSignature = null;</span></p><p><span style="font-family: courier; font-size: x-small;"><br /></span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>// obtain sequence of entities</span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>DerValue[] seq = derReader.getSequence(0);</span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>for (DerValue v : seq)</span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{</span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>if (v.getTag() == 4)</span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{</span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>hashValueFromSignature = v.getOctetString(); // SHA-512 checksum extracted from decrypted signature bytes</span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>}</span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>}</span></p><p><span style="font-family: courier; font-size: x-small;"><br /></span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>MessageDigest md = MessageDigest.getInstance("SHA-512");</span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>md.update(dataBytes);</span></p><p><span style="font-family: courier; font-size: x-small;"><br /></span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>byte[] hashValueCalculated = md.digest();</span></p><p><span style="font-family: courier; font-size: x-small;"><br /></span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>boolean manuallyVerified = Arrays.equals(hashValueFromSignature, hashValueCalculated);</span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>if (manuallyVerified)</span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{</span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>System.out.println("Signature manually verified!");</span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>}</span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>else</span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{</span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>System.out.println("Signature could NOT be manually verified!");</span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>}</span></p><p><span style="font-family: courier; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>}</span></p><p><span style="font-family: courier; font-size: x-small;">}</span></p></div><div><br /></div>Matt (Brisbane/Australia)http://www.blogger.com/profile/16649081553551301779noreply@blogger.com0tag:blogger.com,1999:blog-2959498105239183642.post-71287278695689564062019-10-11T11:01:00.003+10:002021-01-19T14:00:06.234+10:00BLOB BFILE SHA256 calculation from Oracle Database using PL/SQL SQL by way of Java stored procedure<div dir="ltr" style="text-align: left;" trbidi="on">
The code below demonstrates a method to calculate from the Oracle Database an SHA-256 checksum of a BLOB or BFILE by way of a Java stored procedure which in turn can be triggered from a SQL DML statement or PL/SQL block.<br />
<br />
<span style="font-family: "Courier New", Courier, monospace; font-size: x-small;">REM -- we don't want the ampersand in source below interpreted</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">SET DEFINE OFF</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">CREATE OR REPLACE AND RESOLVE JAVA SOURCE NAMED "IOUtils" AS</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">/* MShannon 2019 */</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">import java.io.BufferedInputStream;</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">import java.io.IOException;</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">import java.io.InputStream;</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">import java.security.MessageDigest;</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">import java.security.NoSuchAlgorithmException;</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">import oracle.jdbc.OracleBfile;</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">import oracle.sql.BLOB;</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">public class IO</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">{</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>public static String getSHA256ChecksumHexEncoded(OracleBfile b) throws Exception</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>InputStream is = null;</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>try</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>is = new BufferedInputStream(b.getBinaryStream());</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>return getSHA256ChecksumHexEncoded(is);</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>}</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>finally</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>streamClose(is);</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>}</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>}</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>public static String getSHA256ChecksumHexEncoded(BLOB b) throws Exception</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>InputStream is = null;</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>try</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>is = new BufferedInputStream(b.getBinaryStream());</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>return getSHA256ChecksumHexEncoded(is);</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>}</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>finally</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>streamClose(is);</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>}</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>}</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>public static String getSHA256ChecksumHexEncoded(InputStream is) throws IOException</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>byte[] bytes = getDigest(is, "SHA-256");</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>return toHex(bytes);</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>}</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>public static byte[] getDigest(InputStream is, String algorithm) throws IOException</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>byte[] bytes = new byte[262144];</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>MessageDigest md = null;</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>try</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>md = MessageDigest.getInstance(algorithm);</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>int bytesRead = 0;</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>do</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>bytesRead = is.read(bytes);</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>if (bytesRead > 0)</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>md.update(bytes, 0, bytesRead);</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>}</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>}</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>while (bytesRead != -1);</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>return md.digest();</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>}</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>catch (NoSuchAlgorithmException e)</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>String msg = String.format("Failed to compute the checksum. No such algorithm %s. Error: %s", algorithm,</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>e.getMessage());</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>throw new Error(msg, e);</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>}</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>}</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>public static void streamClose(InputStream in)</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>if (in != null)</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>try</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>in.close();</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>}</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>catch (IOException ignore)</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>}</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>}</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>}</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>public static String toHex(byte[] bytes)</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>if (bytes == null)</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>return null;</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>}</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>StringBuilder sb = new StringBuilder(bytes.length * 2);</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>for (int i = 0; i < bytes.length; i++)</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>{</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>sb.append(Character.forDigit((bytes[i] & 0xf0) >> 4, 16));</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>sb.append(Character.forDigit(bytes[i] & 0x0f, 16));</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>}</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>return sb.toString();</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><span class="Apple-tab-span" style="white-space: pre;"> </span>}</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">}</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">/</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">CREATE OR REPLACE FUNCTION hash_sha256_bfile (p_bfile in BFILE) RETURN VARCHAR2 AS LANGUAGE JAVA</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">NAME 'IO.getSHA256ChecksumHexEncoded(oracle.jdbc.OracleBfile) return String';</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">/</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">CREATE OR REPLACE FUNCTION hash_sha256_blob (p_blob in BLOB) RETURN VARCHAR2 AS LANGUAGE JAVA</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">NAME 'IO.getSHA256ChecksumHexEncoded(oracle.sql.BLOB) return String';</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">/</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">SET DEFINE ON</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<br />
In the example use-case below we calculate an SHA-256 checksum of the file /etc/hosts present on the Database server. We first try SHA-256 checksum calculation direct from a BFILE. We subsequently perform SHA-256 checksum calculation using a BLOB.<br />
<br />
<span style="font-family: "Courier New", Courier, monospace; font-size: x-small;">REM -- AS APPROPRIATE PRIVILEGED USER (e.g DBA) - CREATE DIRECTORY OBJECT</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">CREATE DIRECTORY FILEUPLOADS AS '/etc';</span><br />
<span style="font-family: "Courier New", Courier, monospace; font-size: x-small;">REM -- FOR TESTING ONLY (NOT FOR PRODUCTION) ALLOW EVERYONE TO READ DIR FILES</span><br />
<div>
<span style="font-family: "Courier New", Courier, monospace; font-size: x-small;">GRANT READ ON DIRECTORY FILEUPLOADS TO public;</span></div>
<div>
<br /></div>
<span style="font-family: "Courier New", Courier, monospace; font-size: x-small;">SET SERVEROUTPUT ON</span><br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">DECLARE</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> l_bfile BFILE := BFILENAME('FILEUPLOADS', 'hosts');</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> l_result VARCHAR2(64);</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">BEGIN</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> DBMS_LOB.FILEOPEN(l_bfile, DBMS_LOB.LOB_READONLY);</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> SELECT hash_sha256_bfile(l_bfile) INTO l_result FROM dual;</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> DBMS_OUTPUT.PUT_LINE('Result=' || l_result);</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> -- Close lob objects</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> DBMS_LOB.CLOSE(l_bfile);</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">END;</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">/</span><br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">DECLARE</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> l_bfile BFILE := BFILENAME('FILEUPLOADS', 'hosts');</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> l_blob BLOB;</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> l_result VARCHAR2(64);</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">BEGIN</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> DBMS_LOB.FILEOPEN(l_bfile, DBMS_LOB.LOB_READONLY);</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> DBMS_LOB.CREATETEMPORARY(l_blob,TRUE, DBMS_LOB.SESSION);</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> DBMS_LOB.LOADFROMFILE(</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> dest_lob => l_blob</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> , src_lob => l_bfile</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> , amount => DBMS_LOB.LOBMAXSIZE</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> , dest_offset => 1</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> , src_offset => 1);</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> SELECT hash_sha256_blob(l_blob) INTO l_result FROM dual;</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> DBMS_OUTPUT.PUT_LINE('Result=' || l_result);</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"><br /></span>
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> -- Close lob objects</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> DBMS_LOB.CLOSE(l_bfile);</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;"> DBMS_LOB.FREETEMPORARY(l_blob);</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">END;</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">/</span><br />
<br />
I hope this helps someone!<br />
<div>
<br /></div>
</div>
Matt (Brisbane/Australia)http://www.blogger.com/profile/16649081553551301779noreply@blogger.com0tag:blogger.com,1999:blog-2959498105239183642.post-8889668507438137412015-07-10T11:50:00.001+10:002015-07-10T11:58:58.764+10:00Java SSL HttpUrlConnection Performance Slow using TLS 1.0 with CBC<p><font size="2">The fix Oracle implemented in the JVM to combat the BEAST attack can have a significant performance impact when using TLS 1.0 with CBC. This is particularly noticeable when performing large streaming uploads with HttpURLConnection using the setFixedLength streaming mode (rather than its default mode where it buffers the request payload in full).</font></p> <p><font size="2">When performing writes to HttpURLConnection's OutputStream in setFixedLength streaming mode using a BufferedOutputStream based on the default 8k buffer [<font face="Courier New">OutputStream out = new BufferedOutputStream(uc.getOutputStream())</font>], you can see a pattern like that below when running with the system property <font face="Courier New">-Djavax.net.debug=ssl,handshake</font> set.</font></p> <p><b>Java 6</b> 1.6.0_91<br><small><tt>%% Cached client session: [Session-1, TLS_RSA_WITH_AES_128_CBC_SHA]</tt><tt><br></tt><tt>...</tt><tt><br></tt><tt>main, WRITE: TLSv1 Application Data, length = 32</tt><tt><br></tt><tt>main, WRITE: TLSv1 Application Data, length = 16416</tt><tt><br></tt><tt>main, WRITE: TLSv1 Application Data, length = 32</tt><tt><br></tt><tt>main, WRITE: TLSv1 Application Data, length = 16416</tt><tt><br></tt><tt>...</tt></small><tt><br></tt><br><b>Java 7 </b>1.7.0_15<br><small><tt>%% Cached client session: [Session-1, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA]</tt><tt><br></tt><tt>...</tt><tt><br></tt><tt>main, WRITE: TLSv1 Application Data, length = 32</tt><tt><br></tt><tt>main, WRITE: TLSv1 Application Data, length = 16416</tt><tt><br></tt><tt>main, WRITE: TLSv1 Application Data, length = 32</tt><tt><br></tt><tt>main, WRITE: TLSv1 Application Data, length = 16416</tt><tt><br></tt><tt>...</tt></small></p> <p><small><tt><font size="2" face="Arial">When using Java 8 and TLS 1.2, there are none of the 32 byte packets in the output …</font></tt></small></p> <p><small><tt><font size="2"><font face="Arial"><strong>Java 8 </strong>1.8.0_40</font></font><br><tt>%% Cached client session: [Session-1, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA]</tt><tt><br></tt><tt>...<tt><br></tt>main, WRITE: TLSv1.2 Application Data, length = 16432</tt><tt><br></tt><tt>main, WRITE: TLSv1.2 Application Data, length = 16432</tt><tt><br></tt><tt>main, WRITE: TLSv1.2 Application Data, length = 16432</tt><tt><br></tt><tt>main, WRITE: TLSv1.2 Application Data, length = 16432</tt><br>...<tt><br></tt><br><small><tt><font size="2" face="Arial">If I set the system property "-Djsse.enableCBCProtection=false" with Java 6 (disabling the BEAST attack fix), the 32 byte packets disappear ...</font></tt></small></p> <p><tt>%% Cached client session: [Session-1, TLS_RSA_WITH_AES_128_CBC_SHA]</tt><tt><br></tt><tt>...<tt><br></tt>main, WRITE: TLSv1 Application Data, length = 16416</tt><tt><br></tt><tt>main, WRITE: TLSv1 Application Data, length = 16416</tt><tt><br></tt><tt>main, WRITE: TLSv1 Application Data, length = 16416</tt><tt><br></tt><tt>main, WRITE: TLSv1 Application Data, length = 16416</tt><tt><br></tt><tt>...<tt><br></p><font face="Arial"></font></tt></tt><tt> <p><font size="2" face="Arial">As disabling the CBC protection is not viable in production, I looked at what could be done to minimize the occurrence of the 32 byte packets when using TLS 1.0 with CBC. In turns out by increasing the buffer size of the BufferedOutputStream wrapping HttpURLConnection’s OutputStream from the default 8kb to something much larger e.g. to 256kb, the number of 32 byte packets reduced significantly resulting in a significant performance increase. </font></p> <p><font size="2" face="Arial"><strong>Java 7 </strong>1.7.0_15 with 32k buffer<br><small><tt>%% Cached client session: [Session-1, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA]</tt><tt><br></tt><tt>...</tt><tt><br></tt><tt>main, WRITE: TLSv1 Application Data, length = 32</tt><tt><br></tt><tt>main, WRITE: TLSv1 Application Data, length = 16416</tt><tt><br></tt><tt>main, WRITE: TLSv1 Application Data, length = 16416<tt><br></tt>main, WRITE: TLSv1 Application Data, length = 32</tt><tt><br></tt><tt>main, WRITE: TLSv1 Application Data, length = 16416</tt><tt><br></tt><tt>main, WRITE: TLSv1 Application Data, length = 16416<tt><br></tt>...</tt></small></font></p> <p><font size="2" face="Arial">The larger buffer however as expected had minimal (or no) impact with Java 1.8 based on the TLS 1.2 connection. Java 1.7 can support TLS 1.2, though will by default negotiate TLS 1.0 unless explicitly instructed otherwise:</font></p> <p><a href="http://docs.oracle.com/javase/7/docs/technotes/guides/security/SunProviders.html#tlsprotonote"><u><font color="#000080" size="2" face="Arial">http://docs.oracle.com/javase/7/docs/technotes/guides/security/SunProviders.html#tlsprotonote</font></u></a><br><br><small><tt><i><font size="2">Footnote 1 - Although SunJSSE in the Java SE 7 release supports TLS 1.1 and TLS 1.2, neither version is enabled by default for client connections. Some servers do not implement forward compatibility correctly and refuse to talk to TLS 1.1 or TLS 1.2 clients.</font></i></tt></small></p> <p><small><tt><font size="2" face="Arial">Oracle’s acknowledgement of the BEAST exploit when using TLS 1.0 with CBC (Cipher Block Chaining) is part of CVE-2011-3389:</font></tt></small></p> <p> <table class="texta" border="1" cellspacing="0" summary="Oracle Risk Matrix"> <tbody bgcolor="#ffffff"> <tr> <td style="text-align: center; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px" height="54" valign="center">CVE-2011-3389</td> <td style="text-align: center; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px" valign="center">Java Runtime Environment</td> <td style="text-align: center; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px" valign="center">SSL/TLS</td> <td style="text-align: center; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px" valign="center">JSSE</td> <td style="text-align: center; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px" valign="center">Yes</td> <td style="text-align: center; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px" valign="center">4.3</td> <td style="text-align: center; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px" valign="center">Network</td> <td style="text-align: center; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px" valign="center">Medium</td> <td style="text-align: center; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px" valign="center">None</td> <td style="text-align: center; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px" valign="center">Partial</td> <td style="text-align: center; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px" valign="center">None</td> <td style="text-align: center; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px" valign="center">None</td> <td style="text-align: center; padding-bottom: 1px; padding-left: 1px; padding-right: 1px; padding-top: 1px" valign="center">JDK and JRE 7, 6 Update 27 and before, 5.0 Update 31 and before, 1.4.2_33 and before. <br>JRockit R28.1.4 and before</td></tr></tbody></table><br><i><tt>This is a vulnerability in the SSLv3/TLS 1.0 protocol. Exploitation of this vulnerability requires a man-in-the-middle and the attacker needs to be able to inject chosen plaintext.</tt></i><br><br><font size="2" face="Arial">The links below describe the attack:</font><br><a href="https://blog.torproject.org/blog/tor-and-beast-ssl-attack"><u><font color="#000080" size="2" face="Arial">https://blog.torproject.org/blog/tor-and-beast-ssl-attack</font></u></a><br><a href="http://blogs.cisco.com/security/beat-the-beast-with-tls"><u><font color="#000080" size="2" face="Arial">http://blogs.cisco.com/security/beat-the-beast-with-tls</font></u></a></p> <p><font size="2" face="Arial">To combat the exploit, the fix Oracle did was to split each write() to the underlying OutputStream in to at least two separate TLS records with every record having a different initialization vector. TLS itself caps the maximum record size at 16384 (this is the size of the raw unencrypted bytes). <a href="http://blog.fourthbit.com/2014/12/23/traffic-analysis-of-an-ssl-slash-tls-session">http://blog.fourthbit.com/2014/12/23/traffic-analysis-of-an-ssl-slash-tls-session</a><br>So a write of 16k of client data to the underlying OutputStream at a time with the fix above would result in one TLS record containing the first byte encrypted, and the second TLS record containing the remaining 16383 bytes encrypted. Whereas a write of 32k of client data to the underlying OutputStream at a time would result in three TLS records, one containing the first byte encrypted, the second containing the next 16384 bytes, and the third containing the remaining 16383 bytes encrypted. So when using TLS 1.0 with CBC, the bigger the buffer associated with the write, the fewer one byte encrypted TLS records you are going to see.</font></p> <p><font size="2" face="Arial">To give you an idea of effect that buffer size plays with TLS 1.0 and CBC when the JVM has the fix for BEAST applied:<br>Assuming a file size of 31527359 (~ 30 Megabytes)<br>with 16k buffer: 16384 = 1 + 16383 ; 31527359 / 16384 = ~1924 ; so 1924 one byte ssl records, 1924 x 16383 byte ssl records<br>with 32k buffer: 32768 = 1 + 16384 + 16383; 31527359 / 32768 = ~962 ; so 962 one byte ssl records, 962 x 16384 byte records, and 962 x 16383 byte records<br>with 64k buffer 65536 = 1 + 16384 + 16384 + 16384 + 16383; 31527359 / 65536 = ~481 ; so 481 one byte ssl records, 3*481*16384 byte records, and 481 x 16383 byte records<br>with 256k buffer 262144 = 1 + 15*16384 + 16383; 31527359 / 262144 = ~120; so 120 one byte ssl records, 15*120*16384 byte records, and 120 x 16383 byte records<br>..<br>So to summarize for the 30 megabyte file, buffer size and resulting one-byte ssl records<br>16k: 1924 one byte ssl records<br>32k: 962 one byte ssl records<br>64k: 481 one byte ssl records<br>256k: 120 one byte ssl records<br>512k: 60 one byte ssl records<br>1024k buffer: 30 one byte ssl records</font></p> <p><font size="2" face="Arial">Each SSL record obviously has a reasonable amount of processing time, both client to encrypt/hash, network from a TCP perspective, and server to validate/decrypt the SSL payload.<br>So ideally going forward Java 1.8 using TLS 1.2 is what you want to strive for. If stuck with TLS 1.0, then the large buffer will definitely help with performance.</font></p></tt></tt></small> Matt (Brisbane/Australia)http://www.blogger.com/profile/16649081553551301779noreply@blogger.com0tag:blogger.com,1999:blog-2959498105239183642.post-43391163516454457122014-05-06T19:54:00.001+10:002014-05-06T19:54:27.881+10:00Java split a large file – sample code – high performance<p> </p> <p>Sample Java code to split a source file into chunks.</p> <p>I needed a quick way to split big log files in to manageable chunks that could subsequently be opened with my legacy editor without hitting out-of-memory errors.</p> <p>I did not trust the available freeware solutions HJSplit / FFSJ etc due to the bad VirusTotal.com reports indicating potential malware.</p> <p>So I coded my own using java NIO (Non-Blocking I/O) which provides excellent performance.</p> <p>Source code follows:</p> <p><font color="#0000ff" face="Courier New">import java.io.FileInputStream;<br>import java.io.FileOutputStream;<br>import java.io.IOException;</font></p> <p><font color="#0000ff" face="Courier New">import java.nio.ByteBuffer;</font></p> <p><font color="#0000ff" face="Courier New">import java.nio.channels.FileChannel;</font></p> <p><font color="#0000ff" face="Courier New">/**<br> * Source code to split a file in to chunks using java nio.<br> * <br> * YYYY-MM-DD<br> * 2014-05-06 mshannon - created.<br> */<br>public class Split<br>{<br> public static void main(String[] args) throws IOException<br> {<br> long splitSize = 128 * 1048576; // 128 Megabytes file chunks<br> int bufferSize = 256 * 1048576; // 256 Megabyte memory buffer for reading source file</font></p> <p><font color="#0000ff" face="Courier New"> // String source = args[0];<br> String source = "/C:/Users/mshannon/Desktop/18597996/UCMTRACE/idccs_UCM_server1_1398902885000.log";</font></p> <p><font color="#0000ff" face="Courier New"> // String output = args[1];<br> String output = "/C:/Users/mshannon/Desktop/18597996/UCMTRACE/idccs_UCM_server1_1398902885000.log.split";</font></p> <p><font color="#0000ff" face="Courier New"> FileChannel sourceChannel = null;<br> try<br> {<br> sourceChannel = new FileInputStream(source).getChannel();</font></p> <p><font color="#0000ff" face="Courier New"> ByteBuffer buffer = ByteBuffer.allocateDirect(bufferSize);</font></p> <p><font color="#0000ff" face="Courier New"> long totalBytesRead = 0; // total bytes read from channel<br> long totalBytesWritten = 0; // total bytes written to output</font></p> <p><font color="#0000ff" face="Courier New"> double numberOfChunks = Math.ceil(sourceChannel.size() / (double) splitSize);<br> int padSize = (int) Math.floor(Math.log10(numberOfChunks) + 1);<br> String outputFileFormat = "%s.%0" + padSize + "d";</font></p> <p><font color="#0000ff" face="Courier New"> FileChannel outputChannel = null; // output channel (split file) we are currently writing<br> long outputChunkNumber = 0; // the split file / chunk number<br> long outputChunkBytesWritten = 0; // number of bytes written to chunk so far</font></p> <p><font color="#0000ff" face="Courier New"> try<br> {<br> for (int bytesRead = sourceChannel.read(buffer); bytesRead != -1; bytesRead = sourceChannel.read(buffer))<br> {<br> totalBytesRead += bytesRead;</font></p> <p><font color="#0000ff" face="Courier New"> System.out.println(String.format("Read %d bytes from channel; total bytes read %d/%d ", bytesRead,<br> totalBytesRead, sourceChannel.size()));</font></p> <p><font color="#0000ff" face="Courier New"> buffer.flip(); // convert the buffer from writing data to buffer from disk to reading mode</font></p> <p><font color="#0000ff" face="Courier New"> int bytesWrittenFromBuffer = 0; // number of bytes written from buffer</font></p> <p><font color="#0000ff" face="Courier New"> while (buffer.hasRemaining())<br> {<br> if (outputChannel == null)<br> {<br> outputChunkNumber++;<br> outputChunkBytesWritten = 0;</font></p> <p><font color="#0000ff" face="Courier New"> String outputName = String.format(outputFileFormat, output, outputChunkNumber);<br> System.out.println(String.format("Creating new output channel %s", outputName));<br> outputChannel = new FileOutputStream(outputName).getChannel();<br> }</font></p> <p><font color="#0000ff" face="Courier New"> long chunkBytesFree = (splitSize - outputChunkBytesWritten); // maxmimum free space in chunk<br> int bytesToWrite = (int) Math.min(buffer.remaining(), chunkBytesFree); // maximum bytes that should be read from current byte buffer</font></p> <p><font color="#0000ff" face="Courier New"> System.out.println(<br> String.format(<br> "Byte buffer has %d remaining bytes; chunk has %d bytes free; writing up to %d bytes to chunk",<br> buffer.remaining(), chunkBytesFree, bytesToWrite));</font></p> <p><font color="#0000ff" face="Courier New"> buffer.limit(bytesWrittenFromBuffer + bytesToWrite); // set limit in buffer up to where bytes can be read</font></p> <p><font color="#0000ff" face="Courier New"> int bytesWritten = outputChannel.write(buffer);</font></p> <p><font color="#0000ff" face="Courier New"> outputChunkBytesWritten += bytesWritten;<br> bytesWrittenFromBuffer += bytesWritten;<br> totalBytesWritten += bytesWritten;</font></p> <p><font color="#0000ff" face="Courier New"> System.out.println(<br> String.format(<br> "Wrote %d to chunk; %d bytes written to chunk so far; %d bytes written from buffer so far; %d bytes written in total",<br> bytesWritten, outputChunkBytesWritten, bytesWrittenFromBuffer, totalBytesWritten));</font></p> <p><font color="#0000ff" face="Courier New"> buffer.limit(bytesRead); // reset limit</font></p> <p><font color="#0000ff" face="Courier New"> if (totalBytesWritten == sourceChannel.size())<br> {<br> System.out.println("Finished writing last chunk");</font></p> <p><font color="#0000ff" face="Courier New"> closeChannel(outputChannel);<br> outputChannel = null;</font></p> <p><font color="#0000ff" face="Courier New"> break;<br> }<br> else if (outputChunkBytesWritten == splitSize)<br> {<br> System.out.println("Chunk at capacity; closing()");</font></p> <p><font color="#0000ff" face="Courier New"> closeChannel(outputChannel);<br> outputChannel = null;<br> }<br> }</font></p> <p><font color="#0000ff" face="Courier New"> buffer.clear();<br> }<br> }<br> finally<br> {<br> closeChannel(outputChannel);<br> }<br> }<br> finally<br> {<br> closeChannel(sourceChannel);<br> }</font></p> <p><font color="#0000ff" face="Courier New"> }</font></p> <p><font color="#0000ff" face="Courier New"> private static void closeChannel(FileChannel channel)<br> {<br> if (channel != null)<br> {<br> try<br> {<br> channel.close();<br> }<br> catch (Exception ignore)<br> {<br> ;<br> }<br> }<br> }<br>}<br></font></p> Matt (Brisbane/Australia)http://www.blogger.com/profile/16649081553551301779noreply@blogger.com2tag:blogger.com,1999:blog-2959498105239183642.post-599007777236994212014-02-13T17:29:00.001+10:002014-02-13T17:31:30.417+10:00Two-way SSL guide: Java, Android, Browser clients and WebLogic Server<p align="left">The notes below outline the steps I took to test two-way SSL from scratch using <strong>updated</strong> keytool functionality found in <strong><a href="http://www.oracle.com/technetwork/java/javase/downloads/index.html" target="_blank">Java 7</a></strong>. Rather than use a commercial certificate authority like VeriSign (which costs real money), my notes show how to generate your own CA and all PKI artefacts using just the keytool command. These artefacts can subsequently be utilized for development / testing / private-network scenarios. Note keytool is simple a CLI / console program shipped with the Java JDK / JRE that wraps underling java security/crypto classes.</p> <p>If you can follow these steps and understand the process, then transitioning to a commercial trusted certificate authority like VeriSign should be straightforward.</p> <p>In my previous article <a href="http://todayguesswhat.blogspot.com/2012/07/weblogic-https-one-way-ssl-tutorial.html">http://todayguesswhat.blogspot.com/2012/07/weblogic-https-one-way-ssl-tutorial.html</a> I state:</p> <p><em><font color="#666666">One-way SSL is the mode which most "storefronts" run on the internet so as to be able to accept credit card details and the like without the customer’s details being sent effectively in the clear from a packet-capture perspective. In this mode, the server must present a valid public certificate to the client, but the client is not required to present a certificate to the server. </font></em></p> <p>With Two-way SSL trust is enhanced by now requiring both the server <strong>and the client</strong> present valid certificates to each other so as to prove their identity.</p> <p align="left">From an Oracle WebLogic Server perspective, two-way SSL enables the server to only* accept incoming SSL connections from clients whom can present a public certificate that can be validated based on the contents of the server’s configured trust store.</p> <p align="left">*Assuming <a href="http://docs.oracle.com/cd/E23943_01/apirefs.1111/e13952/taskhelp/security/ConfigureTwowaySSL.html" target="_blank">WebLogic Server</a> is configured with “Client Certs Requested And Enforced” option.</p> <p align="left">The actual certificate verification process itself is quite detailed and would make a good future blog post. RFC specifications of interest are RFC <a href="https://www.ietf.org/rfc/rfc5280.txt" target="_blank">5280</a> (which obsoletes RFC <a href="https://www.ietf.org/rfc/rfc3280.txt" target="_blank">3280</a>) and RFC <a href="https://www.ietf.org/rfc/rfc2818.txt" target="_blank">2818</a> and RFC <a href="https://www.ietf.org/rfc/rfc6125.txt" target="_blank">6125</a>.</p> <p align="left">WebLogic server can also be configured to subsequently authenticate the client based on some attribute (such as cn – common name) extracted from the client’s validated X509 certificate by configuring the Default Identity Asserter; this is commonly known as certificate authentication. This is not mandatory however - Username/password authentication (or any style for that matter) can still be leveraged on top of a two-way SSL connection.</p> <p>Now let’s get on with it …</p> <p><em>Why do we need Java 7 keytool support?</em> Specifically for signing certificate requests, and also to be able to generate a keypair with custom x509 extension such as SubjectAlternativeName / BasicConstraints etc.</p> <p>High-level, we need the following: <br><font color="#9b00d3">Custom Certificate Authority<br>Server Certificate signed by Custom CA<br>Client Certificate signed by Custom CA</font></p> <p>Artifacts required for two-way SSL to support WebLogic server and various clients types (Browser / Java etc):<br><font color="#9b00d3">Server keystore in JKS format<br>Server truststore in JKS format<br>Client keystore in JKS format<br>Client truststore in JKS format<br>Client keystore in PKCS12 keystore format<br>Client truststore in PKCS12 format<br>CA certificate in PEM format</font></p> <p><em>Note: Browsers and Mobile devices typically want public certificates in PEM format and keypairs (private key/public key) in PKCS12 format.<br>Java clients on the other hand generally use JKS format keystores</em></p> <p><u>Steps below assume Linux zsh</u></p> <p><font color="#0000ff" size="2">Constants – edit accordingly</font></p> <p><font color="#4f81bd" face="Courier New">CA_P12_KEYSTORE_FILE=/tmp/ca.p12<br>CA_P12_KEYSTORE_PASSWORD=welcome1<br>CA_KEY_ALIAS=customca<br>CA_KEY_PASSWORD=welcome1<br>CA_DNAME="CN=CustomCA, OU=MyOrgUnit, O=MyOrg, L=MyTown, ST=MyState, C=MyCountry"<br>CA_CER=/tmp/ca.pem</font></p> <p><font color="#4f81bd" face="Courier New">SERVER_JKS_KEYSTORE_FILE=/tmp/server.jks<br>SERVER_JKS_KEYSTORE_PASSWORD=welcome1<br>SERVER_KEY_ALIAS=server<br>SERVER_KEY_PASSWORD=welcome1<br>SERVER_DNAME="CN=www.acme.com" <br>SERVER_CSR=/tmp/server_cert_signing_request.pem<br>SERVER_CER=/tmp/server_cert_signed.pem<br>SERVER_JKS_TRUST_KEYSTORE_FILE=/tmp/server-trust.jks<br>SERVER_JKS_TRUST_KEYSTORE_PASSWORD=welcome1</font></p> <p><font color="#4f81bd" face="Courier New">CLIENT_JKS_KEYSTORE_FILE=/tmp/client.jks<br>CLIENT_JKS_KEYSTORE_PASSWORD=welcome1<br>CLIENT_KEY_ALIAS=client<br>CLIENT_KEY_PASSWORD=welcome1<br>CLIENT_DNAME="CN=mshannon, OU=MyOrgUnit, O=MyOrg, L=MyTown, ST=MyState, C=MyCountry"<br>CLIENT_CSR=/tmp/client_cert_signing_request.pem<br>CLIENT_CER=/tmp/client_cert_signed.pem<br>CLIENT_JKS_TRUST_KEYSTORE_FILE=/tmp/client-trust.jks<br>CLIENT_JKS_TRUST_KEYSTORE_PASSWORD=welcome1<br>CLIENT_P12_KEYSTORE_FILE=/tmp/client.p12<br>CLIENT_P12_KEYSTORE_PASSWORD=welcome1</font></p> <p><font color="#0000ff" size="2">Verify version of Java</font></p> <p><font face="Courier New">(/usr/java/jre/jre1.7.0_45)% <font color="#4f81bd">export JAVA_HOME=`pwd`<br></font>(/usr/java/jre/jre1.7.0_45)% <font color="#4f81bd">export PATH=$JAVA_HOME/bin:$PATH<br></font>(/usr/java/jre/jre1.7.0_45)% <font color="#4f81bd">java -version<br></font><font size="2">java version "1.7.0_45"<br>Java(TM) SE Runtime Environment (build 1.7.0_45-b18)<br>Java HotSpot(TM) 64-Bit Server VM (build 24.45-b08, mixed mode)</font></font></p> <p><font style="background-color: #8064a2"><font color="#ffffff"></font></font><font color="#0000ff" size="2">Create CA, Server and Client keystores</font><font style="background-color: #8064a2"><font color="#ffffff"></font><br></font></p> <p># -keyalg - Algorithm used to generate the public-private key pair - e.g. DSA<br># -keysize - Size in bits of the public and private keys<br># -sigalg - Algorithm used to sign the certificate - for DSA, this would be SHA1withDSA, for RSA, SHA1withRSA<br># -validity - Number of days before the certificate expires<br># -ext bc=ca:true - WebLogic/Firefox require X509 v3 CA certificates to have a Basic Constraint extension set with field CA set to TRUE<br># without the bc=ca:true , firefox won't allow us to import the CA's certificate.</p> <p># look after the CA_P12_KEYSTORE_FILE - it will contain our CA private key and should be locked away!</p> <p><font color="#4f81bd" face="Courier New">keytool -genkeypair -v -keystore "$CA_P12_KEYSTORE_FILE" \<br>-storetype PKCS12 -storepass "$KEYSTORE_PASSWORD" \<br>-keyalg RSA -keysize 1024 -validity 1825 -alias "$CA_KEY_ALIAS" -keypass "$CA_KEY_PASSWORD" -dname "$CA_DNAME" \<br>-ext "bc=ca:true"</font></p> <p><font color="#4f81bd" face="Courier New">keytool -genkeypair -v -keystore "$SERVER_JKS_KEYSTORE_FILE" \<br>-storetype JKS -storepass "$SERVER_JKS_KEYSTORE_PASSWORD" \<br>-keyalg RSA -keysize 1024 -validity 1825 -alias "$SERVER_KEY_ALIAS" -keypass "$SERVER_KEY_PASSWORD" -dname "$SERVER_DNAME"</font></p> <p><font color="#4f81bd" face="Courier New">keytool -genkeypair -v -keystore "$CLIENT_JKS_KEYSTORE_FILE" \<br>-storetype JKS -storepass "$CLIENT_JKS_KEYSTORE_PASSWORD" \<br>-keyalg RSA -keysize 1024 -validity 1825 -alias "$CLIENT_KEY_ALIAS" -keypass "$CLIENT_KEY_PASSWORD" -dname "$CLIENT_DNAME"</font></p> <p><font color="#0000ff" size="2">Export CA certificate</font></p> <p># -rfc - means to output in PEM (rfc style) base64 encoded format, output will look like ----BEGIN.... etc</p> <p><font color="#4f81bd" face="Courier New">keytool -exportcert -v -keystore "$CA_P12_KEYSTORE_FILE" \<br>-storetype PKCS12 -storepass "$KEYSTORE_PASSWORD" \<br>-alias "$CA_KEY_ALIAS" -file "$CA_CER" -rfc</font></p> <p><font color="#0000ff" size="2">Generate certificate signing requests for client and server (to be supplied to CA for subsequent signing)</font></p> <p># The public certificates for the client and server keypairs created above are currently self-signed <br># (such that, issuer = subject , private key signed its associated public certificate)<br># For a production server we want to get our public certificate signed by a valid certificate authority (CA).<br># We are going to use the customca we created above to sign these certificates.<br># We first need to get a certificate signing request ready ...</p> <p><font color="#4f81bd" face="Courier New">keytool -certreq -v -keystore "$SERVER_JKS_KEYSTORE_FILE" \<br>-storetype JKS -storepass "$SERVER_JKS_KEYSTORE_PASSWORD" \<br>-alias "$SERVER_KEY_ALIAS" -keypass "$SERVER_KEY_PASSWORD" -file "$SERVER_CSR"</font></p> <p><font color="#4f81bd" face="Courier New">keytool -certreq -v -keystore "$CLIENT_JKS_KEYSTORE_FILE" \<br>-storetype JKS -storepass "$CLIENT_JKS_KEYSTORE_PASSWORD" \<br>-alias "$CLIENT_KEY_ALIAS" -keypass "$CLIENT_KEY_PASSWORD" -file "$CLIENT_CSR"</font></p> <p><font color="#0000ff" size="2">Sign certificate requests</font></p> <p><font color="#4f81bd" face="Courier New">keytool -gencert -v -keystore "$CA_P12_KEYSTORE_FILE" \<br>-storetype PKCS12 -storepass "$KEYSTORE_PASSWORD" \<br>-validity 1825 -alias "$CA_KEY_ALIAS" -keypass "$CA_KEY_PASSWORD" \<br>-infile "$SERVER_CSR" -outfile "$SERVER_CER" -rfc</font></p> <p><font color="#4f81bd" face="Courier New">keytool -gencert -v -keystore "$CA_P12_KEYSTORE_FILE" \<br>-storetype PKCS12 -storepass "$KEYSTORE_PASSWORD" \<br>-validity 1825 -alias "$CA_KEY_ALIAS" -keypass "$CA_KEY_PASSWORD" \<br>-infile "$CLIENT_CSR" -outfile "$CLIENT_CER" -rfc</font></p> <p><font color="#0000ff" size="2">Import signed certificates</font></p> <p># Once we complete the signing, the certificate is no longer self-signed, but rather signed by our CA.<br># the issuer and owner are now different. </p> <p># Now we are ready to imported the signed public certificates back in to the keystores ...</p> <p># keytool prevents us from importing a certificate should it not be able to verify the full signing chain. <br># As we leveraged our custom CA, we need to import the CA's public certificate in to our keystore as a <br># trusted certificate authority prior to importing our signed certificate.<br># otherwise - 'Failed to establish chain from reply' error will occur</p> <p><font color="#4f81bd" face="Courier New">keytool -import -v -keystore "$SERVER_JKS_KEYSTORE_FILE" \<br>-storetype JKS -storepass "$SERVER_JKS_KEYSTORE_PASSWORD" \<br>-alias "$CA_KEY_ALIAS" -file "$CA_CER" -noprompt</font></p> <p><font color="#4f81bd" face="Courier New">keytool -import -v -keystore "$CLIENT_JKS_KEYSTORE_FILE" \<br>-storetype JKS -storepass "$CLIENT_JKS_KEYSTORE_PASSWORD" \<br>-alias "$CA_KEY_ALIAS" -file "$CA_CER" -noprompt</font></p> <p><font color="#4f81bd" face="Courier New">keytool -import -v -keystore "$SERVER_JKS_KEYSTORE_FILE" \<br>-storetype JKS -storepass "$SERVER_JKS_KEYSTORE_PASSWORD" \<br>-alias "$SERVER_KEY_ALIAS" -keypass "$SERVER_KEY_PASSWORD" -file "$SERVER_CER" -noprompt</font></p> <p><font color="#4f81bd" face="Courier New">keytool -import -v -keystore "$CLIENT_JKS_KEYSTORE_FILE" \<br>-storetype JKS -storepass "$CLIENT_JKS_KEYSTORE_PASSWORD" \<br>-alias "$CLIENT_KEY_ALIAS" -keypass "$CLIENT_KEY_PASSWORD" -file "$CLIENT_CER" -noprompt</font></p> <p><font color="#0000ff" size="2">Create trust stores</font></p> <p># for one-way SSL<br># the trust keystore for the client needs to include the certificate for the trusted certificate authority that signed the certificate for the server</p> <p># for two-way SSL<br># the trust keystore for the client needs to include the certificate for the trusted certificate authority that signed the certificate for the server<br># the trust keystore for the server needs to include the certificate for the trusted certificate authority that signed the certificate for the client</p> <p># for two-way SSL connection, the client verifies the identity of the server and subsequently passes its certificate to the server. <br># The server must then validate the client identity before completing the SSL handshake</p> <p># given our client and server certificates are both issued by the same CA, the trust stores for both will just contain the custom CA cert</p> <p><font color="#4f81bd" face="Courier New">keytool -import -v -keystore "$SERVER_JKS_TRUST_KEYSTORE_FILE" \<br>-storetype JKS -storepass "$SERVER_JKS_TRUST_KEYSTORE_PASSWORD" \<br>-alias "$CA_KEY_ALIAS" -file "$CA_CER" -noprompt</font></p> <p><font color="#4f81bd" face="Courier New">keytool -import -v -keystore "$CLIENT_JKS_TRUST_KEYSTORE_FILE" \<br>-storetype JKS -storepass "$CLIENT_JKS_TRUST_KEYSTORE_PASSWORD" \<br>-alias "$CA_KEY_ALIAS" -file "$CA_CER" -noprompt</font></p> <p><font color="#0000ff" size="2">Create PKCS12 formats for Android / Browser clients</font></p> <p># note - warning will be given that the customca public cert entry cannot be imported<br># TrustedCertEntry not supported<br># this is expected<br># <a href="http://docs.oracle.com/javase/7/docs/technotes/guides/security/crypto/CryptoSpec.html#KeystoreImplementation">http://docs.oracle.com/javase/7/docs/technotes/guides/security/crypto/CryptoSpec.html#KeystoreImplementation</a><br># As of JDK 6, standards for storing Trusted Certificates in "pkcs12" have not been established yet</p> <p><font color="#4f81bd" face="Courier New">keytool -importkeystore -v \<br>-srckeystore "$CLIENT_JKS_KEYSTORE_FILE" -srcstoretype JKS -srcstorepass "$CLIENT_JKS_KEYSTORE_PASSWORD" \<br>-destkeystore "$CLIENT_P12_KEYSTORE_FILE" -deststoretype PKCS12 -deststorepass "$CLIENT_P12_KEYSTORE_PASSWORD" \<br>-noprompt</font></p> <p><font color="#0000ff" size="2">Configuring WebLogic server for two-way SSL</font></p> <p><font color="#4f81bd" face="Courier New">cp "$SERVER_JKS_KEYSTORE_FILE" /u01/app/oracle/product/Middleware/wlserver_10.3/server/lib<br>cp "$SERVER_JKS_TRUST_KEYSTORE_FILE" /u01/app/oracle/product/Middleware/wlserver_10.3/server/lib</font></p> <p>In order to leverage the above keystores, it is just a matter of connecting to the Administration Console, then expand Environment and select Servers.<br>Choose the server for which you want to configure the identity and trust keystores, and select Configuration > Keystores. <br>Change Keystores to be "Custom Identity and Custom Trust"</p> <p>You would then fill in the relevant fields (keystore fully qualified path, type (JKS), and keystore access password). e.g.</p> <p>Custom Identity Keystore: <font face="Courier New">/u01/app/oracle/product/Middleware/wlserver_10.3/server/lib/server.jks<br></font>Custom Identity Keystore Type: <font face="Courier New">JKS</font><br>Custom Identity Keystore Passphrase: <font face="Courier New">welcome1</font></p> <p>Custom Trust Keystore: <font face="Courier New">/u01/app/oracle/product/Middleware/wlserver_10.3/server/lib/server-trust.jks<br></font>Custom Trust Keystore Type: <font face="Courier New">JKS</font><br>Custom Trust Keystore Passphrase: <font face="Courier New">welcome1</font></p> <p>SAVE</p> <p>Next, you need to enable the SSL Listen Port for the server:<br>[Home >Summary of Servers >XXX > ] Configuration > General. </p> <p>SSL Listen Port: Enabled (Check)<br>SSL Listen Port: XXXX</p> <p>SAVE</p> <p>Next, you need to tell WebLogic the alias and password in order to access the private key from the Identity Store:<br>[Home >Summary of Servers >XXX > ] Configuration > SSL. </p> <p>Identity and Trust Locations: Keystores<br>Private Key Alias: <font face="Courier New">server</font><br>Private Key Passphrase: <font face="Courier New">welcome1</font></p> <p>SAVE</p> <p>click Advanced at the bottom of the page ([Home >Summary of Servers >XXX > ] Configuration > SSL_<br>Set the Two Way Client Cert Behaviour attribute to "Client Certs Requested And Enforced"</p> <p> Client Certs Not Requested: The default (meaning one-way SSL).<br> Client Certs Requested But Not Enforced: Requires a client to present a certificate. If a certificate is not presented, the SSL connection continues.<br> Client Certs Requested And Enforced: Requires a client to present a certificate. If a certificate is not presented, the SSL connection is terminated.</p> <p><br>Check "Use JSSE SSL" box<br>Failure to check this box above will likely result in "Cannot convert identity certificate" error when restarting the managed server, and the HTTPS port won't be open for connections.</p> <p>SAVE</p> <p>Restart server.</p> <p><font color="#0000ff" size="2">Configuring Browser client</font></p> <p>From Internet Explorer > Tools > Internet Options > Contents > Certificates</p> <p>Import under the Trusted Root Certification Authorities tab the <font face="Courier New">ca.pem</font> file<br>- Place all certificate in the following store: "Trusted Root Certification Authorities"<br>It will popup Security Warning - You are about to install a certificate from a certification authority (CA) claiming to represent.....<br>Do you want to install this certificate: Yes</p> <p>Import under the Personal tab the <font face="Courier New">client.p12</font> file<br>It will popup with 'To maintain security, the private key was protected with a password'<br>Supply the password for the private key: <font face="Courier New">welcome1</font><br>- Place all certificate in the following store: "Personal"</p> <p>-----------------</p> <p>From Firefox > Tools > Options > Advanced > Encryption > View Certificates<br>From the "Authorities" tab , import <font face="Courier New">ca.pem</font><br>check "Trust this CA to identify websites"<br>ok.<br>(It will get listed under the organization "MyOrg" in the tree based on the certificate dname that was leveraged)<br>From the "Your Certificates" tab, import <font face="Courier New">client.p12</font><br>Enter the password that was used to encrypt this certificate backup: <font face="Courier New">welcome1</font><br>It should state "Successfully restored your security certificate(s) and private key(s)."</p> <p><font color="#0000ff" size="2">Accessing a server configured for two-way SSL from a Browser</font></p> <p>IE - May automatically supply the certificate to the server<br>Firefox - <br>"This site has requested that you identify yourself with a certificate"<br>"Choose a certificate to present as identification"<br>x Remember this decision</p> <p><font color="#0000ff" size="2">Accessing a server configured for two-way SSL from a Java client</font></p> <p>Invoke Java with explicit SSL trust store elements set (to trust server's certificate chain) ...</p> <p><font color="#4f81bd" face="Courier New">System.setProperty("javax.net.ssl.trustStore", "/C:/Users/mshannon/Desktop/client-trust.jks");<br>System.setProperty("javax.net.ssl.trustStoreType", "JKS");<br>System.setProperty("javax.net.ssl.trustStorePassword", "welcome1");</font></p> <p>Otherwise will see...<br><font face="Courier New">"javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target"</font></p> <p>Given we are using two-way SSL, we must also specify the client certificate details ...</p> <p><font color="#4f81bd" face="Courier New">System.setProperty("javax.net.ssl.keyStore", "/C:/Users/mshannon/Desktop/client.jks");<br>System.setProperty("javax.net.ssl.keyStoreType", "JKS");<br>System.setProperty("javax.net.ssl.keyStorePassword", "welcome1");</font></p> <p><font face="Courier New"><font face="Arial">Otherwise will see...<br></font>"javax.net.ssl.SSLHandshakeException: Received fatal alert: bad_certificate"</font></p> <p><strong>Unknown Information </strong>– Is it possible through System properties to specify which key (based on alias) to use from the client keystore, and also can a password be provided for such a key (alias)?<br>Currently we are relying on the fact that the client keystore just contains the one key-pair and that key's alias password entry matches the keystore password!</p><font face="Courier New"> <p><font color="#0000ff" size="2">Configuring WebLogic Server to allow authentication using the client certificate</font></p> <p></font>The above steps should have resulted in two-way SSL transport security. WebLogic however can also be configured to extract the client username from the client supplied public certificate and map this to a user in the identity store. To do this, we need to configure Default Identity Asserter to allow authentication using the client certificate. The deployed web application must also allow client certificate authentication as a login method.</p> <p>Configuring Default Identity Asserter to support X.509 certificates involves:</p> <p>1) Connecting to the WebLogic Server Administration Console as an administrator (e.g. weblogic user)<br>2) Navigating to appropriate security realm (e.g. myrealm)<br>3) Under the Providers tab and Authentication sub-tab selecting DefaultIdentityAsserter (WebLogic Identity Assertion provider)<br>4) On the Settings for DefaultIdentityAsserter page under the Configuration tab and Common sub-tab choosing X.509 as a supported token type and clicking Save.<br>5) On the Settings for DefaultIdentityAsserter page under the Configuration tab and Provider-Specific sub-tab enabling the Default User Name Mapper and choosing the appropriate X.509 attribute within the subject DN to leverage as the username and correcting the delimiter as appropriate and clicking Save.<br>6) Restarting the WebLogic Server.</p> <p>The deployed web application must also list CLIENT-CERT as an authentication method in the web.xml descriptor file:</p> <p><font face="Courier New"><login-config><br> <auth-method>CLIENT-CERT</auth-method><br></login-config></font></p> <p><a href="http://lh4.ggpht.com/-2iLiz4TKr_A/Uvx0UXjWRjI/AAAAAAAAAbc/s-w90t1AwFk/s1600-h/image%25255B20%25255D.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh5.ggpht.com/-Gh1kXUF2ceE/Uvx0VsMqvlI/AAAAAAAAAbk/Balugvdl--s/image_thumb%25255B16%25255D.png?imgmax=800" width="431" height="484"></a> <a href="http://lh3.ggpht.com/-Ern-4nc7z3I/Uvx0XDWq-zI/AAAAAAAAAbs/pxSE8ZMc9Sc/s1600-h/image%25255B24%25255D.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh3.ggpht.com/-sDtRLw8BUDY/Uvx0YfllUEI/AAAAAAAAAb0/EEE34SXcEag/image_thumb%25255B18%25255D.png?imgmax=800" width="431" height="484"></a></p> Matt (Brisbane/Australia)http://www.blogger.com/profile/16649081553551301779noreply@blogger.com1tag:blogger.com,1999:blog-2959498105239183642.post-29641773760429238632013-08-22T14:36:00.001+10:002013-08-22T14:36:25.525+10:00Synology Remote Shutdown Poweroff BusyBox<p>Being the energy conserving environmentalist I am (read – tightass) I look for ways to reduce unnecessary power consumption. I have quite a few devices that are plugged in 24x7, but get limited use. This includes my HTPC setup that comprises a Mac Mini i5 (bootcamp Windows 8) and a Synology DS413j NAS. In the <a href="http://todayguesswhat.blogspot.com.au/2013/08/synology-ds413j-power-consumption.html" target="_blank">previous blog post</a> I outlined costs involved in running the Synology DS413j. The DS413j does not support system hibernation, only disk hibernation. From my testing, it pulls about 11w continuous when the hard disk was hibernated. In the grand scheme of things, this is not a big deal.</p> <p align="left">So what can be done to reduce power consumption? Synology supports scheduled power-on / power-off events through their DSM. There is also the very useful Advanced Power Manager <a href="http://sourceforge.net/projects/advpowermgr4ds/" target="_blank">package</a> that takes this a step further by preventing the shutdown whilst any detected network/disk activity is above a specific threshold. This way, if you are a watching a movie and midway reach the scheduled power-off time, the NAS won’t shutdown until such time disk/network activity falls below the configured threshold.</p> <p align="left"><a href="http://lh4.ggpht.com/-0PoDKb2icME/UhWVKuIA_OI/AAAAAAAAAUg/lfd2wuXpUE8/s1600-h/Advanced%252520Power%252520Manager%25255B4%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="Advanced Power Manager" border="0" alt="Advanced Power Manager" src="http://lh3.ggpht.com/-y-rtzdGlNmM/UhWVL9UeEBI/AAAAAAAAAUo/zeHmuycCRaU/Advanced%252520Power%252520Manager_thumb%25255B2%25255D.png?imgmax=800" width="604" height="480"></a></p> <p>In the screenshot above, I have configured Power Off times for every day of the week, but not necessarily power on times. This means the NAS will always shutdown at night if running, but not necessarily start again in the morning automatically.</p> <p>Whilst the above package is great, I wanted to go a step further and support remote shutdown of the NAS triggered by a desktop shortcut and/or remote control event. In my HTPC setup, I’m using <a href="http://www.eventghost.org" target="_blank">EventGhost</a> to coordinate infrared remote control events to specific actions. I leverage the On Screen Menu EventGhost plugin capability to display a menu of options rendered on my (television) screen for interacting with the HTPC. This includes launching XBMC, Suspending and Shutting down the Mac Mini, Sending a Wake On Lan packet to the Synology. I want to add a new option to this menu to shutdown the NAS.</p> <p>One would think remote shutdown is pretty simple. In fact it can be very simple, by simply Enabling SSH on the NAS, and then leveraging something like putty to make a “root” user connection to the NAS supplying a “Remote command” option like the following</p> <p><a href="http://lh5.ggpht.com/-sYrPY0B0_YE/UhWVMdaGLzI/AAAAAAAAAUw/Ij0B9Xbk6uk/s1600-h/image%25255B2%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; margin: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh6.ggpht.com/-Nk1S3EKRReE/UhWVNGmkMcI/AAAAAAAAAU4/156E3jqcXDk/image_thumb.png?imgmax=800" width="456" height="435"></a></p> <p>You would simply save the putty session as some specific name (e.g root-shutdown), then trigger it using a shortcut link such "C:\Program Files (x86)\PuTTY\PUTTY.EXE" -load "root-shutdown"</p> <p>But what if you want to grant the ability to power off the NAS to some non-root user? Would it not be great to have some user, say the fictitious user “netshutdown”, who simply by connecting to the NAS through telnet / SSH would result in the NAS shutting down?</p> <p>This would be easily accomplished using something like sudo whereby administration commands can easily be delegated. However “sudo” is not available in the standard DSM 4.2 install on the DS413j. Simple then, lets leverage suid on the poweroff executable so that it runs as root. However, the poweroff executable is actually a symbolic link to /bin/busybox. Setting suid on busybox also makes no difference:</p> <p><a href="http://lh6.ggpht.com/-tdlJHVS6rmo/UhWVNn42o1I/AAAAAAAAAVA/pMEsRa9WGtg/s1600-h/image%25255B8%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; margin: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh3.ggpht.com/--wAbi9CXovk/UhWVODcZD3I/AAAAAAAAAVI/yWeVADTovfU/image_thumb%25255B2%25255D.png?imgmax=800" width="670" height="170"></a></p> <p>Setting suid on busybox does however allow “su” to function from a non-root user.</p> <p><a href="http://lh4.ggpht.com/-gI_f0jm5PjA/UhWVOzxzZ4I/AAAAAAAAAVQ/dv5RVorE8l4/s1600-h/image%25255B11%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; margin: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh5.ggpht.com/-XJLuF8iP3oE/UhWVPeAmWTI/AAAAAAAAAVY/TUAFJbPlaNQ/image_thumb%25255B3%25255D.png?imgmax=800" width="527" height="287"></a></p> <p>I’m not comfortable setting suid on the busybox executable given pretty much every command in /bin and a number from /sbin are linked to it. <strong><font size="3"><u>One extreme word of caution!!!</u> </font></strong>Do not make the mistake of executing chmod a-x on the busybox executable or anything linked to it! You will hose your system. I was extremely lucky to have perl installed on my NAS, and had not logged out from my root session, and was able to leverage perl’s chmod function to restore pemissions! (what a relief)</p> <p><a href="http://lh6.ggpht.com/-T3vE3V7LLEA/UhWVPzKlonI/AAAAAAAAAVg/yY3FI5E63R8/s1600-h/image%25255B14%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh3.ggpht.com/-4vIsSIToalA/UhWVQvBbUCI/AAAAAAAAAVo/8yhP2hqHcZU/image_thumb%25255B4%25255D.png?imgmax=800" width="501" height="290"></a></p> <p>If you search for busybox, poweroff, and suid, you find a number of results that talk about employing techniques such as /etc/busybox.conf to call out specific applets and whom can run them, creating c wrapper programs that leverage execve to call busybox, or setuid and system to call /sbin/poweroff etc. I tried all of these and none of them worked with the compiled busybox executable on my NAS; I would receive the Permission denied / Operation not permitted errors.</p> <p>Finally however, I cracked it.</p> <p>I created a wrapper program that rather than call /sbin/poweroff, calls a shell script owned by root, which in turn triggers poweroff.</p> <p>The program (shutdown.c) is as follows:</p> <p><font color="#0000ff" face="Courier New">#include <stdio.h> /* printf function */<br>#include <stdlib.h> /* system function */<br>#include <sys/types.h> /* uid_t type used by setuid */<br>#include <unistd.h> /* setuid function */</font></p> <p><font color="#0000ff" face="Courier New">int main()<br>{<br> printf("Invoking /bin/shutdown.sh ...\n"); <br> setuid(0);<br> system("/bin/shutdown.sh" );<br> return 0;<br>}<br></font></p> <p>/bin/shutdown.sh is as follows:<br><font color="#0000ff" face="Courier New">echo Triggering poweroff command<br>/sbin/poweroff</font></p> <p>I followed the <a href="http://download.synology.com/download/ds/userguide/Synology_DiskStation_Manager_3rd_Party_Apps_Developer_Guide.pdf">developer guide</a> to determine how to compile c programs specific for the NAS. In the case of the DS413j, It leverages a Marvell Kirkwood mv6282 ARMv5te CPU. So I needed to leverage the toolchain for <a href="http://sourceforge.net/projects/dsgpl/files/DSM%204.1%20Tool%20Chains/Marvell%2088F628x%20Linux%202.6.32/">Marvell 88F628x</a>.</p> <p align="left">As I had no native/physical linux machine available for the compilation, I decided to use VirtualBox on Windows and download / leverage the <a href="http://virtualboxes.org/images/lubuntu/">Lubuntu 12.10 VirtualBox image</a> which is a lightweight version of Ubuntu. I set the network card to bridged, started the image, authenticated as lubuntu/lubuntu and updated/added some core packages:</p> <p><font face="Courier New">sudo apt-get update<br>sudo apt-get install build-essential dkms gcc make<br>sudo apt-get install linux-headers-$(uname -r)</font></p> <p><font color="#400040" face="Courier New">sudo -s<br>cd /tmp<br>wget </font><a href="http://sourceforge.net/projects/dsgpl/files/DSM%204.1%20Tool%20Chains/Marvell%2088F628x%20Linux%202.6.32/gcc421_glibc25_88f6281-GPL.tgz"><font color="#400040" face="Courier New">http://sourceforge.net/projects/dsgpl/files/DSM%204.1%20Tool%20Chains/Marvell%2088F628x%20Linux%202.6.32/gcc421_glibc25_88f6281-GPL.tgz</font></a><br><font color="#400040" face="Courier New">tar zxpf gcc421_glibc25_88f6281-GPL.tgz -C /usr/local/</font></p> <p><font color="#400040" face="Courier New">cat > /tmp/shutdown.c <<EOF<br>#include <stdio.h> /* printf function */<br>#include <stdlib.h> /* system function */<br>#include <sys/types.h> /* uid_t type used by setuid */<br>#include <unistd.h> /* setuid function */</font></p> <p><font color="#400040" face="Courier New">int main()<br>{<br> printf("Invoking /bin/shutdown.sh ...\n"); <br> setuid(0);<br> system("/bin/shutdown.sh" );<br> return 0;<br>}<br>EOF</font></p> <p><font color="#400040" face="Courier New">/usr/local/arm-none-linux-gnueabi/bin/arm-none-linux-gnueabi-gcc shutdown.c -o shutdown<br></font><br><a href="mailto:lubuntu@lubuntu-VirtualBox:/tmp$">lubuntu@lubuntu-VirtualBox:/tmp$</a> /usr/local/arm-none-linux-gnueabi/bin/arm-none-linux-gnueabi-gcc shutdown.c -o shutdown<br><a href="mailto:lubuntu@lubuntu-VirtualBox:/tmp$">lubuntu@lubuntu-VirtualBox:/tmp$</a> ls -ltr<br>total 16<br>-rw-rw-r-- 1 lubuntu lubuntu 308 Aug 22 01:29 shutdown.c<br>-rwxrwxr-x 1 lubuntu lubuntu 6715 Aug 22 01:29 shutdown<br><a href="mailto:lubuntu@lubuntu-VirtualBox:/tmp$">lubuntu@lubuntu-VirtualBox:/tmp$</a> <br></p> <p>Now that the shutdown executable was created, I uploaded it to the NAS to the <font face="Courier New">/bin</font> directory. </p> <p>Next I needed the user/group account infrastructure in place on the NAS in order to trigger it. I created a user named “netshutdown” and a group named “shutdown” using the DiskStation Web UI Control Panel User/Group widgets. I also made sure the SSH service was enabled (Control Panel > (Network Services >) Terminal > Enable SSH Service).</p> <p>If you try and SSH leveraging username/password authentication to the NAS as the newly created user, you will see that you are not presented with a shell. This is because Synology locks the user down, which can be seen by viewing the passwd file connected as root:</p> <p><font face="Courier New">media> cat /etc/passwd<br>root:x:0:0:root:/root:/bin/ash<br>lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin<br>ftp:x:21:21:Anonymous FTP User:/nonexist:/sbin/nologin<br>anonymous:x:21:21:Anonymous FTP User:/nonexist:/sbin/nologin<br>smmsp:x:25:25:Sendmail Submission User:/var/spool/clientmqueue:/sbin/nologin<br>postfix:x:125:125:Postfix User:/nonexist:/sbin/nologin<br>dovecot:x:143:143:Dovecot User:/nonexist:/sbin/nologin<br>spamfilter:x:783:1023:Spamassassin User:/var/spool/postfix:/sbin/nologin<br>nobody:x:1023:1023:nobody:/home:/sbin/nologin<br>admin:x:1024:100:System default user:/var/services/homes/admin:/bin/sh<br>guest:x:1025:100:Guest:/nonexist:/bin/sh<br>mshannon:x:1026:100::/var/services/homes/mshannon:/sbin/nologin<br><strong>netshutdown:x:1027:100::/var/services/homes/netshutdown:/sbin/nologin</strong></font><strong></p></strong> <p>Notice the netshutdown user has the shell set as “/sbin/nologin”, and the home directory set to “/var/services/homes/netshudown”. There is no such “homes” directory on my instance.</p> <p>I edited /etc/passwd and changed<br><font face="Courier New">netshutdown:x:1027:100::/var/services/homes/netshutdown:/sbin/nologin<br></font>to<br><font face="Courier New">netshutdown:x:1027:65536::/home/netshutdown:/bin/sh</font></p> <p>65536 is the group id of the new shutdown group:</p> <p><font face="Courier New">media> cat /etc/group<br></font><a href="mailto:#$_@GID__INDEX@_$65536$"><font face="Courier New">#$_@GID__INDEX@_$65536$</font></a><br><font face="Courier New">root:x:0:<br>lp:x:7:lp<br>ftp:x:21:ftp<br>smmsp:x:25:admin,smmsp<br>users:x:100:<br>administrators:x:101:admin<br>postfix:x:125:postfix<br>maildrop:x:126:<br>dovecot:x:143:dovecot<br>nobody:x:1023:<br>shutdown:x:65536:netshutdown</font></p> <p><font face="Courier New"><font face="Arial">I then created the home directory for the user:</font><br>mkdir -p /home/netshutdown<br>chown netshutdown /home/netshutdown</font><font face="Courier New"></p></font> <p>Test it out …</p> <p><font face="Courier New">media> su - netshutdown</font></p> <p><font face="Courier New">BusyBox v1.16.1 (2013-04-16 20:15:54 CST) built-in shell (ash)<br>Enter 'help' for a list of built-in commands.</font></p> <p><font face="Courier New">media> pwd<br>/home/netshutdown</font></p> <p>My user/group was now in place, so it was time to set permissions and configure the shutdown executable and shell script :-</p> <p><font face="Courier New">cd /bin<br>chown root.shutdown shutdown<br>chmod 4750 shutdown</font></p> <p><font face="Courier New">media> ls -la shutdown*<br>-rwsr-x--- 1 root shutdown 6715 Aug 22 09:31 shutdown</font></p> <p><font face="Courier New">cat > /bin/shutdown.sh <<EOF<br>echo Triggering poweroff command<br>/sbin/poweroff<br>EOF</font></p> <p><font face="Courier New">media> ls -la shutdown.sh<br>-rw-r--r-- 1 root root 48 Aug 22 09:35 shutdown.sh</font></p> <p><font face="Courier New">chmod 700 shutdown.sh</font></p> <p><font face="Courier New">media> ls -la shutdown*<br>-rwsr-x--- 1 root shutdown 6715 Aug 22 09:31 shutdown<br>-rwx------ 1 root root 48 Aug 22 09:35 shutdown.sh<br></font></p> <p>For the netshutdown user to automatically trigger the shutdown executable on connection, I had a few different options:</p> <p>Option 1) Change the user’s login shell from <font face="Courier New">/bin/sh</font> to be <font face="Courier New">/bin/shutdown</font><br>Option 2) Create a <font face="Courier New">.profile</font> file for the user, and trigger the <font face="Courier New">/bin/shutdown</font> command from the user’s <font face="Courier New">.profile</font></p> <p>I decided on the latter.</p> <p><font face="Courier New">echo "/bin/shutdown" > /home/netshutdown/.profile<br>chown netshutdown /home/netshutdown/.profile</font></p> <p>Now it was time to test the power off …</p> <p><a href="http://lh6.ggpht.com/-eAWAoDcBFVY/UhWVRLxgPtI/AAAAAAAAAVs/Slzb7hU2DVA/s1600-h/image%25255B17%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; margin: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh4.ggpht.com/-pl3JxgfXMNA/UhWVRq3MLGI/AAAAAAAAAV4/vMISMg4NZ-8/image_thumb%25255B5%25255D.png?imgmax=800" width="565" height="373"></a></p> <p>At this stage, we can now easily create a new putty session that connects via SSH as user netshutdown with username/password authentication. We can invoke that saved session using a shortcut such as PUTTY.EXE -load "XXX Sesion Name”.</p> <p>If you are a sucker for punishment, you can make this thing a bit more complex by utilizing public key authentication (rather than username/password). In the most simple form, you leverage a tool such as puttygen.exe to generate a keypair (private key and public certificate) for a particular user. You enable public key authentication on your NAS, and upload the public certificate of the user to the NAS. You then configure putty to authenticate using public key authentication and point it to the location of your private key. You can go a step further by encrypting your private key so that a passphrase must be supplied on login in order to extract the private key and authenticate. You can also utilize the Putty Authentication Agent (Pageant) to store the decrypted private key in memory, and have putty sessions consult Pageant for the private key on authentication. This <a href="http://chee-yang.blogspot.com.au/2011/07/ssh-public-key-authentication-with.html">blog post</a> does a good job of describing the options.</p> <p>To get public key authentication up and running quickly on your NAS follow steps similar to the following :</p> <p>1) Run puttygen.exe<br>2) Generate new SSH-2 RSA Key without a keyphrase<br>3) Save public key and private key to files (e.g publickey.txt and privatekey.ppk)<br>4) (This should have already been done) From Synology DiskStation UI, Go to Control Panel > (Network Services >) Terminal > Enable SSH Service</p> <p>5) Next SSH using putty.exe to the NAS as the root user</p> <p>6) Edit /etc/ssh/sshd_config</p> <p>uncomment the following two lines<br>#PubkeyAuthentication yes<br>#AuthorizedKeysFile .ssh/authorized_keys</p> <p>and save the file</p> <p>7) Connect as the end-user concerned, and create ~/.ssh directory, and create the file ~/.ssh.ssh/authorized_keys</p> <p>8) Add the public key text from above in to the authorized_keys file and save it ...</p> <p>ssh-rsa AAAAB3Nza......== ....</p> <p>9) Change the permissions <br>chmod 700 ~/.ssh<br>chmod 644 ~/.ssh/authorized_keys</p> <p>10) Open Putty and make the following changes to the session ...</p> <p>Connection type: SSH<br>Connection->Data->Auto-login username: netshutdown<br>Connection->SSH->Auth->Private Key: Your Private Keyfile from above</p> <p>11) Save the session - e.g. netshutdown-media-nas</p> <p>12) Create a new shortcut on your desktop with the following target "C:\Program Files\PuTTY\putty.exe" -load "netshutdown-media-nas"</p> <p>Notes - The private key above contains no passphrase, and is essentially equivalent to having a password in clear text stored in a text file on the desktop. Where security is required, configure the private key with a passphrase. You can then run/utilize Pageant to store effectively the unencrypted private key in memory by supplying the encryption passphrase; this then prevents the continuous password prompting on establishing each new session.</p> Matt (Brisbane/Australia)http://www.blogger.com/profile/16649081553551301779noreply@blogger.com0tag:blogger.com,1999:blog-2959498105239183642.post-17486081036074486512013-08-22T10:28:00.001+10:002013-08-22T10:28:06.845+10:00Synology DS413J Power Consumption<p>I just purchased the entry level Synology DS413j to use for my HTPC media storage. It is running DSM 4.2 3211 The Synology specs state that in HDD hibernation mode, the DS413j should use 7.68 watts. From my testing using an Arlec 240v power meter with just a single 3TB Western Red WD30EFRX installed, the Synology was reading 15w when running, and 11w after HDD hibernation kicked in due to 20 minute disk inactivity. I’m a little annoyed that it is not performing at the advertised 7.68 watts – that is about $6 per year in energy at my tariff of 21.351 cents/kWH. I imagine in order to get that value, Synology must have stripped the NAS down to the bare services required. </p> <p>The DS413 model supposedly can do full system hibernation and drop to 3.37w. If your NAS will 99% of the time be idle, but powered on 24x7, and assuming the 21.351 cents/kWH tariff value from above, you are looking at $6.30 per year to run the DS413 at 3.37w versus $20.57 for the DS413J at 11w:</p> <p>DS413J with only HDD hibernation –> 365 days per year * 24 hours per day * .011 KW * .21351 $/kWH (Origin Energy) = $20.57 per year to run.</p> <p>DS413 with system hibernation –> 365 days per year * 24 hours per day * .00337 KW * .21351 $/kWH (Origin Energy) = $6.30 per year to run.</p> <p>Thus, I could save $14 in a year, by purchasing the DS413 over the DS413J.</p> <p>DS413j = $365 ; DS413 = $519. (519 – 365) / 14 = 11. It would take 11 years for the DS413 to pay for itself on energy savings assuming I never use the NAS :)</p> <p>If on the other hand, you are actively using the NAS 24x7 and fully loaded with 4 disks, you are looking at around 35w for both DS413 and DS413J.</p> <p>365 days per year * 24 hours per day * .035 KW * .21351 $/kWH (Origin Energy) = $65.46 per year to run.</p> Matt (Brisbane/Australia)http://www.blogger.com/profile/16649081553551301779noreply@blogger.com0tag:blogger.com,1999:blog-2959498105239183642.post-42919444261933488352013-05-16T13:03:00.001+10:002013-05-16T13:18:51.388+10:00Unused CSS rules - Optimize HTML pages by removing unreferenced CSS - for free!<p>Google provides some very useful tools and services that can be leveraged to audit web page performance and thus assist in optimizing HTML pages. However these do have their warts.</p> <p><a href="https://developers.google.com/speed/pagespeed/" target="_blank">Page Speed</a> for example provides a free online audit check, as well as Firefox and Chrome browser plugins that can be leveraged to analyse a web page’s performance and assist with optimization. There are two drawbacks with these tools/service</p> <ol> <li>The HTML file must be hosted on a HTTP server and a local file cannot be utilized. Otherwise a message such as the following will occur “<span style="text-transform: none; background-color: rgb(255,255,255); text-indent: 0px; display: inline !important; font: 11px sans-serif; white-space: normal; float: none; letter-spacing: normal; color: rgb(34,34,34); word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px">Unable to run PageSpeed Insights on file:///C:/Users/mshannon/Desktop/test2.html. Please navigate to a HTTP(s) web page.”</span> <li>The browser extensions can be used to “minify” both the HTML and CSS (by removing whitespace / formatting etc), but do not seem to accurately be able to detect unreferenced CSS</li></ol> <p>My solution to the hosted HTML files was to piggyback on dropbox. I created myself a public folder in my dropbox account and uploaded the html and css files. I then right-clicked on the uploaded files and obtained a public link (for example .. https://dl.dropboxusercontent.com/u/12345/test2.html)</p> <p>Once I had the hosted HTML files, I could leverage the Page Speed service and utilize the browser developer tools. The firefox Page Speed extension provides an addon to Firebug. You can see in the image below it has an option to save optimized files to a local location.</p> <p><a href="http://lh3.ggpht.com/-Ayw3c-sULV8/UZRQEdFNDjI/AAAAAAAAATk/fIF90ogrM-c/s1600-h/image%25255B46%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh4.ggpht.com/-RsQ4x4tFYJU/UZRQFCyrxrI/AAAAAAAAATo/XT0tCQp-L-8/image_thumb%25255B30%25255D.png?imgmax=800" width="751" height="400"></a></p> <p>I leveraged the Firefox Page Speed extension purely to get the cleaned/formatted html and css files.</p> <p>What was left now to perform was to remove the unreferenced CSS rules.</p> <p>Google chrome bundles a useful (crippled) developer tool that can be leveraged to audit web page performance and identify unused CSS rules. Unlike Page Speed, this particular tool does a great job of identifying unreferenced rules. It is accessed from Chrome through the following menu/popup combination: Tools > Developer Tools > Audits > Web Page Performance > Run</p> <p><a href="http://lh6.ggpht.com/-D5TVautFavg/UZRMafdUJFI/AAAAAAAAASE/Vxs-OabqRtQ/s1600-h/image%25255B7%25255D.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh4.ggpht.com/-uZxJkKumPR4/UZRMbEs3OPI/AAAAAAAAASM/ZIt0mcoipjk/image_thumb%25255B3%25255D.png?imgmax=800" width="475" height="312"></a></p> <p>You can see by running the tool below, it found a huge number of unreferenced rules:</p> <p><a href="http://lh6.ggpht.com/-XDzmudWlsS4/UZRMbtO3ZzI/AAAAAAAAASU/wBXxSsNhyCU/s1600-h/image%25255B13%25255D.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh3.ggpht.com/-QkePGWVAFlU/UZRMcQQh9VI/AAAAAAAAASc/5P97zMx7hBQ/image_thumb%25255B7%25255D.png?imgmax=800" width="475" height="312"></a></p> <p>The problem with this tool however, is that it does not provide a mechanism to write out the good (referenced/used) CSS rules.</p> <p>A bit of searching around, and I found this excellent post:</p> <p><a href="http://dev.bootstraponline.com/2011/11/automatically-remove-unused-css-rules.html">http://dev.bootstraponline.com/2011/11/automatically-remove-unused-css-rules.html</a></p> <p>Essentially, the dude had patched the chrome developer tools to write out the good css rules when running the audit. The problem for me though, was this was done using Linux builds, and also I wasn’t sure whether newer builds of chrome would offer better detection capabilities.</p> <p>So I set about getting the latest windows chrome builds from http://commondatastorage.googleapis.com/chromium-browser-continuous/index.html</p> <p>What I found however, is that they stopped shipping devtools_frontend.zip at build 158804. At the time of writing this, they were now up to build 200347. Additionally, the contents of devtools_frontend.zip with build 158804 did not seem to match the files that boostraponline referenced in his/her patch. So I went trawling backwards in builds until I found the very last version to ship with AuditRules.js inside devtools_frontend.zip. This was build 152197 which is circa August 2012.</p> <p>You need the following two files:</p> <p><a href="http://commondatastorage.googleapis.com/chromium-browser-continuous/Win/152197/chrome-win32.zip">http://commondatastorage.googleapis.com/chromium-browser-continuous/Win/152197/chrome-win32.zip</a></p> <p><a href="http://commondatastorage.googleapis.com/chromium-browser-continuous/Win/152197/devtools_frontend.zip">http://commondatastorage.googleapis.com/chromium-browser-continuous/Win/152197/devtools_frontend.zip</a></p> <p>Next …</p> <ol> <li>extract build 152197 of chrome-win32.zip (e.g. to desktop) ; this automatically creates a folder named "chrome-win32" <li>extract build 152197 of devtools_frontend.zip to a *new* folder named "devtools_frontend" under "chrome-win32"; the "devtools_frontend" folder needs to be created <li>create a new folder named "user-data" under "chrome-win32" <li>create a run.bat file to trigger execution of chrome similar to the following:</li></ol> <p><font face="Courier New">@echo off<br>set CHROMEDIR=C:\Users\mshannon\Desktop\chrome-win32<br>%CHROMEDIR%\chrome.exe --user-data-dir=%CHROMEDIR%\user-data --debug-devtools-frontend=%CHROMEDIR%\devtools_frontend</font></p> <p>Execute the bat file and see if chrome successfully loads. Once confirmed, close down chrome.</p> <p>The patch diff that bootstraponline provides is mostly accurate. The one change though is you must now utilize “InspectorFrontendHost.save("used.css", usedCss, true);”</p> <p>You can download the patched AuditRules.js file for the above build 152197 from my dropbox: <a href="http://db.tt/5ZUFkIqX">http://db.tt/5ZUFkIqX</a></p> <p>Simple replace devtools_frontend\AuditRules.js with the file above.</p> <p>Having performed the patch, fire up chrome from the bat file. Invoke the audit tool against the web page (or local file) and you will be presented with a save-as box should it detect unreferenced CSS rules. The file to be saved (used.css) contains the rules that were referenced.</p> <p><a href="http://lh3.ggpht.com/-nqCpjWDdUg4/UZRQFgBcbqI/AAAAAAAAATw/ZUkhh-aow_8/s1600-h/image%25255B47%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh4.ggpht.com/-DgTCxb3j6GE/UZRQGceCnNI/AAAAAAAAAT8/R8K6XOTSb0w/image_thumb%25255B31%25255D.png?imgmax=800" width="668" height="292"></a></p> <p>Thanks bootstraponline and Google!</p> Matt (Brisbane/Australia)http://www.blogger.com/profile/16649081553551301779noreply@blogger.com0tag:blogger.com,1999:blog-2959498105239183642.post-88985371148243891262013-04-13T17:42:00.001+10:002013-04-13T17:42:04.352+10:00Recover / Decrypt Weblogic password from boot.properties<p><font size="2">When installing a Weblogic domain in development mode, the Configuration wizard will generate a boot identity file for the administration server containing the encrypted username and password of the initial administrative user. These credentials are then automatically leveraged when starting the admin server and avoid the need for the weblogic administrator to manually supply these. It is also possible to utilize a boot identify file (boot.properties) in production domains. See the following link for more information: </font><a href="http://docs.oracle.com/cd/E14571_01/web.1111/e13708/overview.htm#i1068887"><font size="2">http://docs.oracle.com/cd/E14571_01/web.1111/e13708/overview.htm#i1068887</font></a></p> <p><font size="2">Recovering/decrypting a credential value from the boot identity file is reasonably straightforward should you have shell and executable access to the Weblogic installation.</font></p> <p><font size="2">First, obtain the DOMAIN_HOME value …</font></p> <p><font color="#0000ff" size="2" face="Courier New">ps auxwww | grep Name=AdminServer | tr " " "\n" | grep "domain.home"</font></p> <p><font color="#646b86" size="2" face="Courier New">-Ddomain.home=/u01/app/oracle/product/Middleware/user_projects/domains/base_domain</font></p> <p>Next, source the setDomainEnv.sh file …</p> <p><font color="#0000ff" size="2" face="Courier New">export DOMAIN_HOME=/u01/app/oracle/product/Middleware/user_projects/domains/base_domain</font></p> <p><font color="#0000ff" size="2" face="Courier New">source $DOMAIN_HOME/bin/setDomainEnv.sh</font></p> <p><font size="2">Extract the encrypted username and password credential from the boot identify file ...</font></p> <p><font color="#0000ff" size="2" face="Courier New">USR=`grep username $DOMAIN_HOME/servers/AdminServer/security/boot.properties | sed -e "s/^username=\(.*\)/\1/"`</font></p> <p><font color="#0000ff" size="2" face="Courier New">PW=`grep password $DOMAIN_HOME/servers/AdminServer/security/boot.properties | sed -e "s/^password=\(.*\)/\1/"`</font></p> <p><font size="2">Sample values …</font></p> <p><font color="#646b86" size="2" face="Courier New">mshannon@slc05elc%</font><font color="#646b86" size="2" face="Courier New"> echo $USR<br>{AES}RI+L8BLQQc3mTwbCx59un+vcHJ4c30GMQ90ovDY7VLI=</font></p> <p><font color="#646b86" size="2" face="Courier New">mshannon@slc05elc%</font><font color="#646b86" size="2" face="Courier New"> echo $PW<br>{AES}B9acQuaVUBNqsem1FzGROqu7w2tqZenm3StwYB3C+bM=</font></p> <p><font size="2">Create the small java Decrypt program and invoke it supplying the DOMAIN_HOME and encrypted value requiring decryption …</font></p> <p><font color="#0000ff" size="2" face="Courier New">cat > /tmp/Decrypt.java <<EOF<br>public class Decrypt {<br> public static void main(String[] args) {<br> System.out.println("Decrypted value: " + new weblogic.security.internal.encryption.ClearOrEncryptedService(<br> weblogic.security.internal.SerializedSystemIni.getEncryptionService(args[0])).<br> decrypt(args[1]));<br> }<br>}<br>EOF</font></p> <p><font color="#0000ff" size="2" face="Courier New">$JAVA_HOME/bin/javac -d /tmp /tmp/Decrypt.java</font></p> <p><font color="#0000ff" size="2" face="Courier New">$JAVA_HOME/bin/java -cp /tmp:$CLASSPATH Decrypt "$DOMAIN_HOME" "$USR"</font></p> <p><font color="#0000ff" size="2" face="Courier New">$JAVA_HOME/bin/java -cp /tmp:$CLASSPATH Decrypt "$DOMAIN_HOME" "$PW"</font></p> <p><font face="Courier New"></font><font size="2">Sample output …</font> </p> <p><font color="#646b86" size="2" face="Courier New">mshannon@slc05elc%</font><font color="#646b86" size="2" face="Courier New"> $JAVA_HOME/bin/java -cp /tmp:$CLASSPATH Decrypt "$DOMAIN_HOME" "$USR"<br>Decrypted value: weblogic<br></font><font size="2" face="Courier New"></font></p> <p><font color="#646b86" size="2" face="Courier New">mshannon@slc05elc%</font><font color="#646b86" size="2" face="Courier New"> $JAVA_HOME/bin/java -cp /tmp:$CLASSPATH Decrypt "$DOMAIN_HOME" "$PW"<br>Decrypted value: welcome1</font></p> Matt (Brisbane/Australia)http://www.blogger.com/profile/16649081553551301779noreply@blogger.com4tag:blogger.com,1999:blog-2959498105239183642.post-19309637890178646902013-03-27T11:39:00.001+10:002013-03-27T11:39:34.531+10:00Simple Chrome black background dark theme extension - How to build one in 5 minutes<p>There are times when a white background is best replaced with a black one. If you are using Chrome, you can build your own custom extension in just a few minutes that can easily modify colours of your favourite website(s). To do this, we leverage the <a href="http://developer.chrome.com/extensions/content_scripts.html" target="_blank">Content Scripts</a> feature available to Chrome Extensions that is kind of like a poor man’s version of GreaseMonkey. Best of all though, there are no closed custom 3rd party Chrome extensions to install from some unknown developer, here you will be the developer!</p> <p>Content Scripts provide a mechanism to manipulate the DOM. The DOM is essentially a tree of all objects that constitute the webpage (images / links / text / styles etc).</p> <p>What we are going to do is create an unpacked extension that simply adds a new stylesheet link at the end of the page load which will change the background colour to black, and the text/link/visited-link colours to various shades of grey. For our test, we will modify all pages falling under cnn.com and yahoo.com.</p> <p>To get started, first create yourself a folder (on your Desktop or wherever) that will host the two files that comprise our extension, for example "<font size="2" face="Courier New">Black Background Extension</font>"</p> <p>Within this folder, create a file named <font size="2" face="Courier New"><font color="#ff0000">toggle.js</font> <font face="Arial">that has the following contents:</font></font></p> <p><font color="#0000ff" face="Courier New">var cssStyle='* { background: black !important; color: #EEEEEE !important }'<br>+ ' :link, :link * { color: #A1A1A1 !important }'<br>+ ' :visited, :visited * { color: #505050 !important }'; </font></p> <p><font color="#0000ff" face="Courier New">if(document.createStyleSheet) { <br> document.createStyleSheet("javascript:'"+cssStyle+"'"); <br>} else {<br> var cssLink = document.createElement('link');<br> cssLink.rel = 'stylesheet';<br> cssLink.href = 'data:text/css,'+escape(cssStyle);<br> document.getElementsByTagName("head")[0].appendChild(cssLink);<br>}</font></p> <p>Next, create a file named <font color="#ff0000" size="2" face="Courier New">manifest</font><font size="2" face="Courier New"><font color="#ff0000">.json</font> <font face="Arial">that has the following contents:</font></font></p> <p><font color="#0000ff" face="Courier New">{<br> "name": "Black Background",<br> "version": "1.0",<br> "description": "Sets background colour to black, and text, link, and visited link to shades of grey",<br> "manifest_version": 2,<br> "content_scripts": [<br> {<br> "matches": ["<a href="http://*.cnn.com/">http://*.cnn.com/</a>*", "<a href="http://*.yahoo.com/">http://*.yahoo.com/</a>*"],<br> "js": ["toggle.js"],<br> "run_at": "document_end",<br> "all_frames": true<br> }<br> ]<br>}</font></p> <p><font color="#0000ff" face="Courier New"><font color="#444444" face="Arial"> Now, it is a simple matter of firing up Chrome, and typing in <font face="Courier New">chrome://extensions</font> in the address bar. Once the Extensions are displayed, activate the <em>Developer mode</em> option.</font></font></p> <p><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh3.ggpht.com/-gXhXSnFD2s0/UVJNyIb8R7I/AAAAAAAAAQw/FDNvL6OF7vM/image%25255B22%25255D.png?imgmax=800" width="644" height="217"></p><font color="#0000ff" face="Courier New"><font color="#444444" face="Arial"> <p><font color="#0000ff" face="Courier New"><font color="#444444" face="Arial"> Next, choose the Load unpacked extension… button and navigate to the folder created above hosting our unpacked extension.</font></font></p> <p><a href="http://lh3.ggpht.com/-pkI4of-EEDg/UVJNys3-nhI/AAAAAAAAAQ4/2dG_EE14Gn4/s1600-h/image%25255B21%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh4.ggpht.com/-3x1xT-ec20k/UVJNzsIjEjI/AAAAAAAAARA/wRltK0gQqO8/image_thumb%25255B11%25255D.png?imgmax=800" width="644" height="217"></a></p> <p><font color="#0000ff" face="Courier New"><font color="#444444" face="Arial">If all goes to plan, our extension should now be listed. We also have the option of packing our extension in to a signed zip file that would allow us to redistribute it.</font></font></p> <p><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh5.ggpht.com/-jlhbfHRgFpA/UVJN0RtpP1I/AAAAAAAAARI/JWWkPWpm1Z4/image%25255B20%25255D.png?imgmax=800" width="644" height="217"></p> <p><font color="#0000ff" face="Courier New"><font color="#444444" face="Arial">Finally, simply restart the Chrome browser and attempt to access a site from our match rules (cnn.com for example). You should see that the background colour is now black!</font></font></p> <p><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh6.ggpht.com/-PDlGX1IDafA/UVJN1KqvgfI/AAAAAAAAARQ/T4EW6cKyIYM/image%25255B19%25255D.png?imgmax=800" width="244" height="197"></p> <p>For details on the content_scripts options leveraged in our manifest, refer to the Chrome documentation <a href="http://developer.chrome.com/extensions/content_scripts.html" target="_blank">http://developer.chrome.com/extensions/content_scripts.html</a></p> <p> </p></font></font> Matt (Brisbane/Australia)http://www.blogger.com/profile/16649081553551301779noreply@blogger.com0tag:blogger.com,1999:blog-2959498105239183642.post-65060898761354635632013-03-20T12:03:00.001+10:002013-03-20T12:03:31.291+10:00How to determine if an Oracle LOB is stored as a SECUREFILE or BASICFILE<p><font size="2">The <font face="Courier New">DESCRIBE</font> command on an Oracle table is not sufficient to determine whether a LOB column is stored as a SECUREFILE or a regular old BASICFILE. Instead you must query <font face="Courier New">USER_LOBS</font> (or <font face="Courier New">DBA_LOBS</font> etc), or alternatively leverage the PL/SQL <font face="Courier New">dbms_lob.issecurefile</font> function.</font></p> <p><font face="Courier New">% sqlplus</font></p> <p><font face="Courier New">SQL*Plus: Release 11.2.0.1.0 Production on Tue Mar 19 18:04:53 2013</font></p> <p><font face="Courier New">Copyright (c) 1982, 2009, Oracle. All rights reserved.</font></p> <p><font face="Courier New">Enter user-name: / as sysdba</font></p> <p><font face="Courier New">Connected to:<br>Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production<br>With the Partitioning, OLAP, Data Mining and Real Application Testing options</font></p> <p><font face="Courier New">SQL> show parameter compatible;</font></p> <p><font face="Courier New">NAME TYPE VALUE<br>------------------------------------ ----------- ------------------------------<br>compatible string 11.2.0.0.0<br></font></p> <p><font face="Courier New">SQL> show parameter db_securefile;</font></p> <p><font face="Courier New">NAME TYPE VALUE<br>------------------------------------ ----------- ------------------------------<br>db_securefile string PERMITTED</font></p> <p><font face="Courier New"></font> </p> <p><font color="#0000ff" size="2">The DB_SECUREFILE parameter specifies whether or not to treat LOB files as SecureFiles by default.</font></p> <p><font color="#0000ff" size="2">NEVER - LOBs that are specified as SecureFiles are created as BasicFile LOBs. Any SecureFile-specific storage options specified will result in an exception. <br>PERMITTED - LOBs are allowed to be created as SecureFiles, but will be created as BasicFile by default.<br>ALWAYS - All LOBs created in the system are created as SecureFile LOBs.<br>IGNORE - The SECUREFILE keyword and all SecureFile options are ignored.</font></p> <p><font color="#0000ff" size="2">If the COMPATIBLE parameter is not set to 11.1 or higher, then LOBs are not treated as SecureFiles.</font></p> <p><font face="Courier New"></font> </p> <p><font face="Courier New">SQL> create user matt identified by welcome1;</font></p> <p><font face="Courier New">User created.</font></p> <p><font face="Courier New">SQL> grant create session to matt;</font></p> <p><font face="Courier New">Grant succeeded.</font></p> <p><font face="Courier New">SQL> grant create table to matt;</font></p> <p><font face="Courier New">Grant succeeded.</font></p> <p><font face="Courier New">SQL> grant unlimited tablespace to matt;</font></p> <p><font face="Courier New">Grant succeeded.</font></p> <p><font face="Courier New">SQL> conn matt/welcome1<br>Connected.</font></p> <p><font face="Courier New">SQL> <br>CREATE TABLE test(<br> lob1 BLOB<br> ,lob2 BLOB<br> ,lob3 BLOB<br> ,lob4 BLOB<br>) <br> LOB(lob2) STORE AS BASICFILE<br> ,LOB(lob3) STORE AS SECUREFILE<br> ,LOB(lob4) STORE AS SECUREFILE (<br> ENABLE STORAGE IN ROW<br> NOCACHE LOGGING<br> COMPRESS MEDIUM<br> DEDUPLICATE<br> )<br>/</font></p> <p><font face="Courier New">Table created.</font></p><font face="Courier New"> <p><br><font color="#0000ff" size="2" face="Arial">Logging options:</font></p> <p><font color="#0000ff" size="2" face="Arial">LOGGING - LOB changes generate full entries in redo logs<br>NOLOGGING - LOB changes are not logged in the redo logs and cannot be replayed in the event of failure.</font></p> <p><font color="#0000ff" size="2" face="Arial">Caching options</font></p> <p><font color="#0000ff" size="2" face="Arial">CACHE - LOB data is placed in the buffer cache.<br>CACHE READS - LOB data is only placed in the buffer cache only during read operations but not during write operations.<br>NOCACHE - LOB data is not placed in the buffer cache, or brought in to the buffer cache and placed at the least recently used end of the LRU list.</font></p> <p><font color="#0000ff" size="2" face="Arial">SecureFile LOBs also support FILESYSTEM_LIKE_LOGGING logging option which is similar to metadata journaling of file systems</font></p> <p> </p> <p>SQL> desc test;<br> Name Null? Type<br> ----------------------------------------- -------- ----------------------------<br> LOB1 BLOB<br> LOB2 BLOB<br> LOB3 BLOB<br> LOB4 BLOB</p> <p>SQL> desc user_lobs;<br> Name Null? Type<br> ----------------------------------------- -------- ----------------------------<br> TABLE_NAME VARCHAR2(30)<br> COLUMN_NAME VARCHAR2(4000)<br> SEGMENT_NAME VARCHAR2(30)<br> TABLESPACE_NAME VARCHAR2(30)<br> INDEX_NAME VARCHAR2(30)<br> CHUNK NUMBER<br> PCTVERSION NUMBER<br> RETENTION NUMBER<br> FREEPOOLS NUMBER<br> CACHE VARCHAR2(10)<br> LOGGING VARCHAR2(7)<br> ENCRYPT VARCHAR2(4)<br> COMPRESSION VARCHAR2(6)<br> DEDUPLICATION VARCHAR2(15)<br> IN_ROW VARCHAR2(3)<br> FORMAT VARCHAR2(15)<br> PARTITIONED VARCHAR2(3)<br> SECUREFILE VARCHAR2(3)<br> SEGMENT_CREATED VARCHAR2(3)</p> <p> </p> <p>set linesize 100</p> <p>col Column format a6<br>col isSecureFile format a12<br>col Compressed format a10<br>col DeDuplicated format a12<br>col Encrypted format a9<br>col StoredInRow format a11<br>col Logging format a7<br>col Cached format a10</p> <p>SELECT<br> column_name as "Column"<br> ,securefile as "isSecureFile"<br> ,compression as "Compressed"<br> ,deduplication as "DeDuplicated"<br> ,encrypt as "Encrypted"<br> ,in_row as "StoredInRow"<br> ,logging as "Logging"<br> ,cache as "Cached"<br>FROM user_lobs<br>WHERE table_name = 'TEST'</p> <p><br>Column isSecureFile Compressed DeDuplicated Encrypted StoredInRow Logging Cached<br>------ ------------ ---------- ------------ --------- ----------- ------- ----------<br>LOB1 NO NONE NONE NONE YES YES NO<br>LOB2 NO NONE NONE NONE YES YES NO<br>LOB3 YES NO NO NO YES YES NO<br>LOB4 YES MEDIUM LOB NO YES YES NO</p> <p> </p> <p>SQL></p> <p>insert into test values(empty_blob(), empty_blob(), empty_blob(), empty_blob())</p> <p>set serveroutput on</p> <p>DECLARE<br> l1 BLOB; l2 BLOB; l3 BLOB; l4 BLOB;<br>BEGIN<br> SELECT lob1, lob2, lob3, lob4<br> INTO l1, l2, l3, l4<br> FROM test<br> WHERE rownum = 1;</p> <p> IF dbms_lob.issecurefile(l1) THEN<br> dbms_output.put_line('Stored in a securefile');<br> ELSE<br> dbms_output.put_line('Not stored in a securefile');<br> END IF;</p> <p> IF dbms_lob.issecurefile(l2) THEN<br> dbms_output.put_line('Stored in a securefile');<br> ELSE<br> dbms_output.put_line('Not stored in a securefile');<br> END IF;</p> <p> IF dbms_lob.issecurefile(l3) THEN<br> dbms_output.put_line('Stored in a securefile');<br> ELSE<br> dbms_output.put_line('Not stored in a securefile');<br> END IF;</p> <p> IF dbms_lob.issecurefile(l4) THEN<br> dbms_output.put_line('Stored in a securefile');<br> ELSE<br> dbms_output.put_line('Not stored in a securefile');<br> END IF;</p> <p>END;<br>/</p> <p><br>Not stored in a securefile<br>Not stored in a securefile<br>Stored in a securefile<br>Stored in a securefile</p> <p>PL/SQL procedure successfully completed.</font></p> Matt (Brisbane/Australia)http://www.blogger.com/profile/16649081553551301779noreply@blogger.com0tag:blogger.com,1999:blog-2959498105239183642.post-80779916347022758852013-03-06T11:10:00.001+10:002013-03-06T11:10:54.005+10:00Disable Adobe Reader XI (11.x) Welcome Screen<p> </p> <p>Create and execute a registry file (disablewelcomescreen.reg) with contents:</p> <p><font color="#0000ff" face="Courier New">REGEDIT4</font></p> <p><font color="#0000ff" face="Courier New">[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Adobe\Acrobat Reader\11.0\FeatureLockDown\cWelcomeScreen]<br>"bShowWelcomeScreen"=dword:00000000</font></p> <p>Alternatively, open <strong>regedit</strong> and then navigate to Computer > HKEY_LOCAL_MACHINE > SOFTWARE > Policies > Adobe > Acrobat Reader > 11.0 > FeatureLockDown.</p> <p>right click on FeatureLockDown and choose New > Key.</p> <p>name the key <strong>cWelcomeScreen</strong></p> <p>right click the cWelcomeScreen key and choose New > DWORD (32-bit) Value</p> <p>name the DWORD value <strong>bShowWelcomeScreen</strong></p> <p>leave the value as 0</p> Matt (Brisbane/Australia)http://www.blogger.com/profile/16649081553551301779noreply@blogger.com0tag:blogger.com,1999:blog-2959498105239183642.post-9159574898318775362013-02-26T12:46:00.001+10:002013-02-26T12:46:17.410+10:00xargs - is command executed once or multiple times – examples<p><strong><font face="Courier New">xargs</font></strong> is used to execute a command one or potentially multiple times using the standard input as arguments to that command.</p> <p>What sometimes in unclear is whether the command invoked by xargs is executed once or multiple times.</p> <p>This blog post should clear things up …</p> <p>Firstly, here is the xargs program usage in layman’s terms</p> <p><font face="Courier New">xargs [xargs options] [command] [arguments for command based on xargs standard input]</font></p> <p>The standard input would typically be output that was piped from an earlier command such as find / ls / echo etc. It doesn’t have to be though, you can invoke xargs as a standalone no-arg command and simply type the input and terminate with a control-d to signal end-of-terminal.</p> <p>Let’s try a simple example combining the xargs --verbose option so that we can see the command-line that xargs will execute on the standard error output before it is actually invoked. In the screenshot below, I invoke the xargs command, and then enter 1 to 5 separating each with a newline (enter), followed by a terminating ctrl-d. As I provided the --verbose option to xargs, it wrote out the command that it will execute and the arguments that it is going to provide to that command.</p> <p><a href="http://lh5.ggpht.com/-IBtX0l3IyY8/USwhyIe8qYI/AAAAAAAAAO0/A7Vqvq9bRK0/s1600-h/image%25255B2%25255D.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh5.ggpht.com/-700fw00__ss/USwhzJoEaLI/AAAAAAAAAO8/ssbhPw4mjOw/image_thumb.png?imgmax=800" width="184" height="153"></a></p> <p>The output above shows that be default if you don’t provide an explicit command for xargs to invoke, it will leverage /bin/echo. You can also see xargs invoked this echo command just a single time. What may not be obvious is how xargs processed the standard input to come up with arguments to supply the (in this case 'echo') command. xargs will by default treat whitespace and newlines as delimiters. In the example below, I entered: 1 TAB 2 NEWLINE 3 SPACE 4 NEWLINE 5 NEWLINE ctrl-d. </p> <p><a href="http://lh3.ggpht.com/-QhnzPAvzy2g/USwh0RjljAI/AAAAAAAAAPE/_VLaf-NHfsc/s1600-h/image%25255B5%25255D.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh4.ggpht.com/-6BrfnJ2wV7s/USwh1J-vu3I/AAAAAAAAAPM/5h-FRWqGIR4/image_thumb%25255B1%25255D.png?imgmax=800" width="179" height="131"></a></p> <p><strong>So how and under what conditions does the command that xargs executes get invoked multiple times?</strong></p> <p>Well.. you can either explicitly tell xargs that a particular command can only operate on a specific number of arguments at a time, and that the command should be reinvoked as required to work through the remaining arguments; OR … xargs may determine itself that the command to execute along with any arguments hits a maximum command-line length, in which case it automatically splits the arguments across multiple command invocations.</p> <p>Let’s first explicitly tell xargs that a command invocation should only work on a maximum number of arguments at a time. You do this through the -n option …</p> <p><a href="http://lh5.ggpht.com/-aI3YdUzfDwM/USwh13O9WtI/AAAAAAAAAPU/l_0upjRa8Ms/s1600-h/image%25255B11%25255D.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh4.ggpht.com/-L7nsB5-VUi0/USwh2yLwOtI/AAAAAAAAAPc/Gp_58vMIoXo/image_thumb%25255B5%25255D.png?imgmax=800" width="208" height="566"></a></p> <p>The above example first shows supplying a “-n1” option which results in a command being invocated for each argument. Later the “-n2” option is leveraged which result in a command being invoked for every two arguments. The final xargs test above shows how 1 TAB 2 NEWLINE 3 SPACE 4 NEWLINE 5 NEWLINE ctrl-d is processed with the “-n1” option. You can see that xargs immediately starts invoking commands after each line of input is processed.</p> <p>As mentioned prior, xargs may itself automatically split arguments across multiple commands if a maximum command-line character length is reached. The xargs binary will likely have a default size limit hardcoded to the operating system ARG_MAX length.</p> <p><a href="http://lh5.ggpht.com/-ovhhvK-WxSI/USwh3TCPgPI/AAAAAAAAAPk/_OV1RBsOn8w/s1600-h/image%25255B14%25255D.png"><img style="background-image: none; border-right-width: 0px; margin: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh5.ggpht.com/-IpOPV_IdWNc/USwh3zYNuRI/AAAAAAAAAPs/UrN4dyNYJ6Y/image_thumb%25255B6%25255D.png?imgmax=800" width="169" height="34"></a></p> <p>You can explicitly tell xargs the max command-line length using the --max-chars or -s options. The length comprises the command and initial-arguments and the terminating nulls at the ends of the argument strings. As seen below “/bin/echo 1” will take 12 chars, and “/bin/echo 1 2” will take 14 chars (including terminating null).</p> <p><font size="3" face="Courier New">/bin/echo 1 2<br>12345678901234</font></p> <p>Let’s try out this “-s” option…</p> <p><a href="http://lh4.ggpht.com/-Ky7CEaEhJKM/USwh49uwEKI/AAAAAAAAAP0/KwhNnAdlWes/s1600-h/image%25255B18%25255D.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh3.ggpht.com/-5vjrohzn_BY/USwh5k5w5tI/AAAAAAAAAP8/mwQMre_J2Lc/image_thumb%25255B8%25255D.png?imgmax=800" width="387" height="611"></a></p> <p>Here is a final example that ties everything together. It demonstrates the following:</p> <ol> <li>xargs receiving piped standard-input from a prior command (i.e tr / find). <li>xargs using the --verbose option to output the command that will be invoked <li>xargs being told to leverage an explicit delimiter “-d” option, e.g “\n” – newline <li>xargs invoking an explicit command (the Unix “file” command) <li>xargs invoking an explicit command once per argument (“-n1”) <li>find command leveraging the “-print0” option to delimit search results by the null character (rather than newline), in conjunction with the “-0” (or --null) xargs option so that any search results which contain whitespace are treated correctly as a single command-argument.</li></ol> <p><a href="http://lh5.ggpht.com/-ISvq4qYsf6w/USwh6u5idiI/AAAAAAAAAQE/jsVPUBy03k0/s1600-h/image%25255B22%25255D.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh6.ggpht.com/-mK9L6rg_gXs/USwh7YlQt_I/AAAAAAAAAQM/FLzKh7SyG9A/image_thumb%25255B10%25255D.png?imgmax=800" width="591" height="595"></a></p> <p><a href="http://lh5.ggpht.com/-ZB8Bojk_Cx0/USwh8XSXk7I/AAAAAAAAAQU/isJ9AElDxEg/s1600-h/image%25255B26%25255D.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh3.ggpht.com/-Qlk-wPfvsVs/USwh9bYNMRI/AAAAAAAAAQc/7Ay7hFugSIE/image_thumb%25255B12%25255D.png?imgmax=800" width="594" height="264"></a></p> Matt (Brisbane/Australia)http://www.blogger.com/profile/16649081553551301779noreply@blogger.com0tag:blogger.com,1999:blog-2959498105239183642.post-28997365831426514772013-02-06T18:31:00.001+10:002013-02-06T18:31:36.889+10:00Java Applet terminated in browser when debugging – solution<p>I recently had to troubleshoot an old applet that was not working as expected. I intended to leverage JDeveloper to debug the applet utilizing JPDA socket debug steps listed in: <a href="http://docs.oracle.com/javase/6/docs/technotes/guides/plugin/developer_guide/debugger.html">http://docs.oracle.com/javase/6/docs/technotes/guides/plugin/developer_guide/debugger.html</a></p> <p>I was able to successfully connect/attach to the VM from JDeveloper and start stepping through code, however within a minute or so, the applet would unexpectedly terminate. You can imagine how frustrating this was. A quick search on google listed possible causes such as browser plugin watchdogs, need for administration privileges, JRE bugs etc. etc.</p> <p>It turns out the problem was caused due to a breakdown in the heartbeat messages sent between the browser VM and client VM. Invoking the browser with the environment variable <strong>JPI_PLUGIN2_NO_HEARTBEAT=1</strong> enabled the debugging to proceed as intended without the applet/Java-plugin being killed / terminating.</p> <p>Below are the steps I performed to successfully debug an applet:</p> <p>From Windows Control Panel choose Java > "Java" tab > "View..." button for Java Runtime Environment Settings<br>Double click table column value for Runtime Parameters for the user JRE entry.</p> <p>Set the value as <br><font face="Courier New">-agentlib:jdwp=transport=dt_socket,server=y,address=8453,suspend=n</font></p> <p>click "OK"<br>Click "Apply" on the Java Runtime Environment Settings page<br>Click "OK" on the Java Runtime Environment Settings page</p> <p><a href="http://lh3.ggpht.com/-hD1IkBrMV8c/URIU2stiKfI/AAAAAAAAAOI/QI0z9yfWqxM/s1600-h/jre%25255B4%25255D.jpg"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="jre" border="0" alt="jre" src="http://lh6.ggpht.com/-Uz_x6TGPVl0/URIU30ggEeI/AAAAAAAAAOQ/qvEfcF4PvJs/jre_thumb%25255B2%25255D.jpg?imgmax=800" width="567" height="614"></a><br>For debugging, the following advanced options should also be set for the JRE:</p> <p>"Advanced" tab > Settings > Debugging<br>Enable tracing<br>Enable logging<br></p> <p>"Advanced" tab > Settings > Java console<br>Choose Show console<br></p> <p>Having configured the JRE options, we now need to disable that heartbeat message mechanism before we launch the browser. As mentioned above, failure to perform this will result in the Applet / JRE abnormally terminating typically under a minute or so of stepping in to code.<br>You will likely see an exception along the lines of <br>basic: JVM[id=0]-Heartbeat heartbeat dead, exception. dT=NNN seconds.</p> <p>From command-prompt<br><font face="Courier New">set JPI_PLUGIN2_NO_HEARTBEAT=1</font></p> <p>And then launch browser, e.g. </p> <p><font face="Courier New">%LOCALAPPDATA%\Google\Chrome\Application\chrome.exe --disable-hang-monitor</font></p> <p><font face="Courier New"><em>e.g. C:\Users\mshannon\AppData\Local\Google\Chrome\Application\chrome.exe --disable-hang-monitor</em></font></p> <p>The "<strong>--disable-hang-monitor</strong>" option passed to Chrome prevents it from popping up the ‘<u>plugin not responding</u>’ dialog.</p> <p><a href="http://lh5.ggpht.com/-S9G493sLoAQ/URIU4uvc_9I/AAAAAAAAAOY/tbpcWWuHtSY/s1600-h/image%25255B3%25255D.png"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh5.ggpht.com/-gsNQ-t6VjaY/URIU5tao58I/AAAAAAAAAOg/3eXiqL1ilGo/image_thumb%25255B1%25255D.png?imgmax=800" width="778" height="407"></a></p> <p>Additional command-line options for Browser JVM Troubleshooting can also be set; These result in verbose console messages such as rendering the parameters supplied to the applet.</p> <p><font face="Courier New">set JPI_PLUGIN2_DEBUG=1<br>set JPI_PLUGIN2_VERBOSE=1</font></p> <p>Check for JRE logs/traces in <font face="Courier New">%USERPROFILE%\AppData\LocalLow\Sun\Java\Deployment\log</font></p> <p><font face="Courier New"><em>e.g. C:\Users\mshannon\AppData\LocalLow\Sun\Java\Deployment\log<br></em></font></p> <p>Once I was able to properly step through code, I found the cause of the problem with our applet. The applet was no longer able to obtain from the browser an OAM 11g webgate cookie which had been set to be httponly.</p> Matt (Brisbane/Australia)http://www.blogger.com/profile/16649081553551301779noreply@blogger.com0tag:blogger.com,1999:blog-2959498105239183642.post-17755655430596900412013-02-06T17:08:00.001+10:002013-02-06T17:10:24.219+10:00Simple Solution - Black Wallpaper Background in Android<p>I’ve just got hold of an Android Samsung Galaxy S2 phone, and I want a fixed background color, in particular black. Why does Google make this so difficult.</p> <p>Here is my solution that involves installing nil/zero android apps that took less than a minute.</p> <p>Step 1. Go to GSMARENA.COM and lookup the Display size resolution for your phone. For the galaxy S2, it is 480 x 800 pixels.</p> <p>Step 2. Fire up a basic paint application, such as Windows paint.</p> <p>Step 3. Resize the image to the resolution from above</p> <p>Step 4. Choose the appropriate color you want for the wallpaper/background, and leverage the fill/bucket tool to fill the image</p> <p>Step 5. Save the image as a GIF, JPG, or PNG</p> <p>Step 6. Email the image to your gmail account accessible from your android phone</p> <p>Step 7.Open Gmail application and the email containing image just sent</p> <p>Step 8. Click Save and View on Image</p> <p>Step 9. Press Menu button in Viewer, and Choose “Set as” option and Home screen wallpaper or Lock screen wallpaper etc.</p> <p>Step 10. Choose the correct orientation for the image – portrait etc, and click Save.</p> <p>Congratulation, you should now have a fixed background.</p> <p><a href="http://lh5.ggpht.com/-vHnDC_VQU0k/URIBQSTBpSI/AAAAAAAAAMk/2IYA71MHazs/s1600-h/Snap1%25255B2%25255D.jpg"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="Snap1" border="0" alt="Snap1" src="http://lh6.ggpht.com/-1jiNJDQmN-I/URIBRL2nK6I/AAAAAAAAAMs/ypibWfYAgLg/Snap1_thumb.jpg?imgmax=800" width="244" height="213"></a><a href="http://lh6.ggpht.com/-ZZAi1CXUEWM/URIBR6BYaDI/AAAAAAAAAM0/-7ai_rLp0Yc/s1600-h/Snap2%25255B2%25255D.jpg"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="Snap2" border="0" alt="Snap2" src="http://lh4.ggpht.com/-QzRO59DPhME/URIBSg9DC-I/AAAAAAAAAM8/pFBzg1xsyW4/Snap2_thumb.jpg?imgmax=800" width="170" height="244"></a><a href="http://lh4.ggpht.com/-d9SMO8t0XpA/URIBTAuBeEI/AAAAAAAAANE/vNEUfX_ud6k/s1600-h/Snap3%25255B2%25255D.jpg"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="Snap3" border="0" alt="Snap3" src="http://lh6.ggpht.com/-rGnmaR_wxm4/URIBTmZZL3I/AAAAAAAAANM/CrQVPW4_82I/Snap3_thumb.jpg?imgmax=800" width="244" height="105"></a><a href="http://lh3.ggpht.com/-_z797Oa8J_0/URIBUfPLCRI/AAAAAAAAANU/-T4TMHag0Lo/s1600-h/Snap4%25255B2%25255D.jpg"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="Snap4" border="0" alt="Snap4" src="http://lh3.ggpht.com/-BlAmKikJf38/URIBVIloJ0I/AAAAAAAAANc/-my1woxN-rk/Snap4_thumb.jpg?imgmax=800" width="226" height="244"></a></p> <p>If you have a 480x800 phone, you can probably just save the image linked below.</p> <p><a href="http://lh4.ggpht.com/-RitI9S9kjiM/URIBVonTmzI/AAAAAAAAAN0/Jy-FxFT2GMg/s1600-h/black%25255B9%25255D.gif"><img style="background-image: none; border-bottom: 0px; border-left: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top: 0px; border-right: 0px; padding-top: 0px" title="black" border="0" alt="black" src="http://lh4.ggpht.com/-bzLSxaATmcA/URIBWPJxOTI/AAAAAAAAAN4/HwK33L1ktlw/black_thumb%25255B7%25255D.gif?imgmax=800" width="480" height="800"></a></p> Matt (Brisbane/Australia)http://www.blogger.com/profile/16649081553551301779noreply@blogger.com0tag:blogger.com,1999:blog-2959498105239183642.post-9250307398531316982012-09-15T01:01:00.001+10:002012-09-15T09:08:06.216+10:00JAXB - XSD to Java Map/HashMap example using xjc, bindings and XmlAdapter<p>I’ve been assigned the task of implementing for my specific product team a common RESTful API that is invoked as part of a cloud on-boarding process. The spec provided to me describes the data structures comprising the request and responses – all of which will be encoded in JSON. In an ideal world, I would leverage something like JAX-RS (using something like Jersey RI). But, alas, that would be too easy. Instead I must host this API somehow on top of my product’s existing service-based architecture framework. I went searching around for a JAXB equivalent for JSON – that would allow some type of JSON to Java binding. I was hoping I would find some type of JSON schema definition concept, an xjc and schemgen equivalent etc. I came up short on my search for such tools, but I did discover that <a href="http://wiki.fasterxml.com/JacksonJAXBAnnotations">Jackson</a> and Jersey can support de/serialization from/to JSON of Java objects that are annotated using JAXB (java.xml.bind.annotation). This was a welcome discovery and it meant I could set about trying to model the specification’s data structures with XSD. </p> <p>My plan was to create the XML schema up front, and then leverage xjc to create the set of JAXB-annotated Java classes that map to the elements/types defined in the schema. Everything was coming along nicely. Whenever I got stuck on the XSD front, I would simply try and model the concept using some basic java classes, then fire up the schemagen tool to view the schema it generated, and incorporate the techniques/result back in to my own XSD. I completed the XSD, and invoked xjc and out came my auto-generated JAXB-annotated classes. However, some of the classes and properties were not what I was expecting. Where had my java.util.Map based properties gone?</p> <p>If you take a simple sample like the following – a Person object with a map property of all their worldly gadgets (iPad/iPhones etc):</p> <p><u>Person.java</u></p> <p><font color="#0000ff" face="Courier New">import java.util.Map;</font></p> <p><font color="#0000ff" face="Courier New">public class Person<br>{<br> private String name;<br> private Map<String, Gadget> gadgets;</font></p> <p><font color="#0000ff" face="Courier New"> public void setName(String name)<br> {<br> this.name = name;<br> }</font></p> <p><font color="#0000ff" face="Courier New"> public String getName()<br> {<br> return name;<br> }</font></p> <p><font color="#0000ff" face="Courier New"> public void setGadgets(Map<String, Gadget> gadgets)<br> {<br> this.gadgets = gadgets;<br> }</font></p> <p><font color="#0000ff" face="Courier New"> public Map<String, Gadget> getGadgets()<br> {<br> return gadgets;<br> }<br>}</font></p> <p><u>Gadget.java</u></p> <p><font color="#0000ff" face="Courier New">public class Gadget<br>{<br> private String make;</font></p> <p><font color="#0000ff" face="Courier New"> public void setMake(String make)<br> {<br> this.make = make;<br> }</font></p> <p><font color="#0000ff" face="Courier New"> public String getMake()<br> {<br> return make;<br> }<br>} </font></p> <p>Running schemagen on the above, you get the following schema:</p> <p><font color="#646b86" face="Courier New"><?xml version="1.0" encoding="UTF-8" standalone="yes"?><br><xs:schema version="1.0" xmlns:xs="</font><a href="http://www.w3.org/2001/XMLSchema"><font color="#646b86" face="Courier New">http://www.w3.org/2001/XMLSchema</font></a><font color="#646b86" face="Courier New">"></font></p> <p><font color="#646b86" face="Courier New"> <xs:complexType name="gadget"><br> <xs:sequence><br> <xs:element name="make" type="xs:string" minOccurs="0"/><br> </xs:sequence><br> </xs:complexType></font></p> <p><font color="#646b86" face="Courier New"> <xs:complexType name="person"><br> <xs:sequence><br> <xs:element name="gadgets"><br> <xs:complexType><br> <xs:sequence><br> <xs:element name="entry" minOccurs="0" maxOccurs="unbounded"><br> <xs:complexType><br> <xs:sequence><br> <xs:element name="key" minOccurs="0" type="xs:string"/><br> <xs:element name="value" minOccurs="0" type="gadget"/><br> </xs:sequence><br> </xs:complexType><br> </xs:element><br> </xs:sequence><br> </xs:complexType><br> </xs:element><br> <xs:element name="name" type="xs:string" minOccurs="0"/><br> </xs:sequence><br> </xs:complexType><br></xs:schema></font></p> <p>If I then feed the above schema back to xjc, the JAXB-annotated classes that get generated look like the following (javadoc removed to save on space):</p> <p><u>Person.java</u></p> <p><font color="#0000ff" face="Courier New">import java.util.ArrayList;<br>import java.util.List;<br>import javax.xml.bind.annotation.XmlAccessType;<br>import javax.xml.bind.annotation.XmlAccessorType;<br>import javax.xml.bind.annotation.XmlElement;<br>import javax.xml.bind.annotation.XmlType;</font></p> <p><font color="#0000ff" face="Courier New">@XmlAccessorType(XmlAccessType.FIELD)<br>@XmlType(name = "person", propOrder = {<br> "gadgets",<br> "name"<br>})<br>public class Person {</font></p> <p><font color="#0000ff" face="Courier New"> @XmlElement(required = true)<br> protected Person.Gadgets gadgets;<br> protected String name;</font></p> <p><font color="#0000ff" face="Courier New"> public Person.Gadgets getGadgets() {<br> return gadgets;<br> }</font></p> <p><font color="#0000ff" face="Courier New"> public void setGadgets(Person.Gadgets value) {<br> this.gadgets = value;<br> }</font></p> <p><font color="#0000ff" face="Courier New"> public String getName() {<br> return name;<br> }</font></p> <p><font color="#0000ff" face="Courier New"> public void setName(String value) {<br> this.name = value;<br> }</font></p><font color="#0000ff" face="Courier New"> <p><br> @XmlAccessorType(XmlAccessType.FIELD)<br> @XmlType(name = "", propOrder = {<br> "entry"<br> })<br> public static class Gadgets {</p> <p> protected List<Person.Gadgets.Entry> entry;</p> <p> public List<Person.Gadgets.Entry> getEntry() {<br> if (entry == null) {<br> entry = new ArrayList<Person.Gadgets.Entry>();<br> }<br> return this.entry;<br> }</p> <p> @XmlAccessorType(XmlAccessType.FIELD)<br> @XmlType(name = "", propOrder = {<br> "key",<br> "value"<br> })<br> public static class Entry {</p> <p> protected String key;<br> protected Gadget value;</p> <p> public String getKey() {<br> return key;<br> }</p> <p> public void setKey(String value) {<br> this.key = value;<br> }</p> <p> public Gadget getValue() {<br> return value;<br> }</p> <p> public void setValue(Gadget value) {<br> this.value = value;<br> }</p> <p> }</p> <p> }</p> <p>}</font></p> <p><u>Gadget.java</u></p> <p><font color="#0000ff" face="Courier New">import javax.xml.bind.annotation.XmlAccessType;<br>import javax.xml.bind.annotation.XmlAccessorType;<br>import javax.xml.bind.annotation.XmlType;</font></p> <p><font color="#0000ff" face="Courier New">@XmlAccessorType(XmlAccessType.FIELD)<br>@XmlType(name = "gadget", propOrder = {<br> "make"<br>})<br>public class Gadget {</font></p> <p><font color="#0000ff" face="Courier New"> protected String make;</font></p> <p><font color="#0000ff" face="Courier New"> public String getMake() {<br> return make;<br> }</font></p> <p><font color="#0000ff" face="Courier New"> public void setMake(String value) {<br> this.make = value;<br> }<br>}<br></font></p> <p><font color="#0000ff" face="Courier New"><font color="#000000" face="Arial">As can be seen above in the generated Person class, the Map was not reinstated. Instead a new Person.Gadgets class was created containing a list of the new Person.Gadgets.Entry class. To be fair, the xjc tool treated the schema at face value. How was it to know that this structure should be modelled by a Map. Ideally, it would be nice if only a few instructions in a bindings file (supplied to xjc tool along with the schema file) were sufficient to auto-generate all required JAXB-annotated classes with full Map support. Unfortunately this is not the case. Instead a bindings file must be created that targets appropriate elements in the schema and overrides their baseType with a fully-qualified custom Map subclass. Custom java files must be hand-created for the Map subclass, and also an XmlAdapter subclass which contains the logic to unmarshal/marshal to/from the Map subclass.</font></font></p> <hr> <p><font color="#000000">What proceeds is a fully-worked example based on the Person / Gadget scenario above that restores Map support for set/getGadgets() methods of the Person class. It also has a few extra features thrown in including subclassing of Gadget. Credit for this technique must go to Aaron Anderson @ adventuresintechology.blogspot.com.</font></p> <p><font color="#0000ff" face="Courier New"><font color="#000000" face="Arial">First, the xml schema.</font></font></p> <p><font color="#0000ff" face="Courier New"><font color="#000000" face="Arial"><u>schema.xsd</u></font></font></p> <p><font color="#646b86" face="Courier New"><?xml version="1.0" encoding="UTF-8"?><br><xs:schema xmlns:xs="</font><a href="http://www.w3.org/2001/XMLSchema"><font color="#646b86" face="Courier New">http://www.w3.org/2001/XMLSchema</font></a><font color="#646b86" face="Courier New">" elementFormDefault="qualified"<br>targetNamespace="</font><a href="http://todayguesswhat.blogspot.com/test"><font color="#646b86" face="Courier New">http://todayguesswhat.blogspot.com/test</font></a><font color="#646b86" face="Courier New">" xmlns:test="</font><a href="http://todayguesswhat.blogspot.com/test"><font color="#646b86" face="Courier New">http://todayguesswhat.blogspot.com/test</font></a><font color="#646b86" face="Courier New">"></font></p> <p><font color="#646b86" face="Courier New"><xs:element name="Person"><br> <xs:complexType><br> <xs:sequence><br> <xs:element name="name" type="xs:string"/><br> <xs:element name="gadgets" type="test:GadgetMapModeller" minOccurs="0"/><br> </xs:sequence><br> </xs:complexType><br></xs:element></font></p> <p><font color="#646b86" face="Courier New"><xs:complexType name="GadgetMapModeller"><br> <xs:sequence><br> <xs:element name="entry" minOccurs="0" maxOccurs="unbounded"><br> <xs:complexType><br> <xs:sequence><br> <xs:element name="key" type="xs:string"/><br> <xs:element name="value" type="test:Gadget"/><br> </xs:sequence><br> </xs:complexType><br> </xs:element><br> </xs:sequence><br></xs:complexType></font></p> <p><font color="#646b86" face="Courier New"><xs:complexType name="Gadget"><br> <xs:sequence><br> <xs:element name="make" type="xs:string"/><br> <xs:element name="model" type="xs:string"/><br> <xs:element name="year" type="xs:int"/><br> </xs:sequence><br></xs:complexType></font></p> <p><font color="#646b86" face="Courier New"><xs:complexType name="Computer"><br> <xs:complexContent><br> <xs:extension base="test:Gadget"><br> <xs:sequence><br> <xs:element name="speed" type="xs:int"/><br> <xs:element name="cpu" type="xs:string"/><br> </xs:sequence><br> </xs:extension><br> </xs:complexContent><br></xs:complexType></font></p> <p><font color="#646b86" face="Courier New"></xs:schema></font></p> <p>The important bindings file that overrides the generated type of the gadgets element to the new Map subclass – GadgetMap<String, Gadget></p> <p><u>bindings.xml</u></p> <p><font color="#646b86" face="Courier New"><jaxb:bindings xmlns:jaxb="</font><a href="http://java.sun.com/xml/ns/jaxb"><font color="#646b86" face="Courier New">http://java.sun.com/xml/ns/jaxb</font></a><font color="#646b86" face="Courier New">" xmlns:xs="</font><a href="http://www.w3.org/2001/XMLSchema"><font color="#646b86" face="Courier New">http://www.w3.org/2001/XMLSchema</font></a><font color="#646b86" face="Courier New">" version="2.1"></font></p> <p><font color="#646b86" face="Courier New"><jaxb:bindings schemaLocation="schema.xsd"><br> <br> <jaxb:bindings node="//xs:element[@name='Person']//xs:element[@name='gadgets']"><br> <jaxb:property><br> <jaxb:baseType name="com.blogspot.todayguesswhat.test.model.GadgetMap&lt;String,Gadget&gt;" /><br> </jaxb:property><br> </jaxb:bindings></font></p> <p><font color="#646b86" face="Courier New"></jaxb:bindings></font></p> <p><font color="#646b86" face="Courier New"></jaxb:bindings></font></p> <p>The new Map subclass with XmlJavaTypeAdapter JAXB annotation defining the name of the adapter for unmarshalling/marshalling to/from this type.</p> <p><u>GadgetMap.java</u></p> <p><font color="#0000ff" face="Courier New">package com.blogspot.todayguesswhat.test.model;</font></p> <p><font color="#0000ff" face="Courier New">import java.util.HashMap;</font></p> <p><font color="#0000ff" face="Courier New">import javax.xml.bind.annotation.adapters.XmlJavaTypeAdapter;</font></p> <p><font color="#0000ff" face="Courier New">@XmlJavaTypeAdapter(GadgetMapAdapter.class)<br>public class GadgetMap<String,Gadget> extends HashMap<String,Gadget><br>{ <br>}</font></p> <p>The XmlAdapter that does the important conversion from one type to another: </p> <p><u>GadgetMapAdapter.java</u></p> <p><font color="#0000ff" face="Courier New">package com.blogspot.todayguesswhat.test.model;</font></p> <p><font color="#0000ff" face="Courier New">import java.util.Map;</font></p> <p><font color="#0000ff" face="Courier New">import javax.xml.bind.annotation.adapters.XmlAdapter;</font></p> <p><font color="#0000ff" face="Courier New">public class GadgetMapAdapter extends XmlAdapter<GadgetMapModeller, GadgetMap<String,Gadget>><br>{ <br> @Override<br> public GadgetMap<String,Gadget> unmarshal(GadgetMapModeller modeller)<br> {<br> GadgetMap<String,Gadget> map = new GadgetMap<String,Gadget>();<br> for (GadgetMapModeller.Entry e : modeller.getEntry())<br> {<br> map.put(e.getKey(), e.getValue());<br> }<br> return map;<br> }</font></p> <p><font color="#0000ff" face="Courier New"> @Override<br> public GadgetMapModeller marshal(GadgetMap<String,Gadget> map)<br> {<br> GadgetMapModeller modeller = new GadgetMapModeller();<br> for (Map.Entry<String,Gadget> entry : map.entrySet())<br> {<br> GadgetMapModeller.Entry e = new GadgetMapModeller.Entry();<br> e.setKey(entry.getKey());<br> e.setValue(entry.getValue());<br> modeller.getEntry().add(e);<br> }<br> return modeller;<br> }<br>}</font></p> <p>A few test classes.</p> <p>MarshalTest.java</p> <p><font color="#0000ff" face="Courier New">package com.blogspot.todayguesswhat.test.model;<br> <br>import java.util.List;</font></p> <p><font color="#0000ff" face="Courier New">import javax.xml.bind.JAXBContext;<br>import javax.xml.bind.Marshaller;<br> <br>public class MarshalTest<br>{<br> public static void main(String[] args) throws Exception<br> {<br> JAXBContext jc = JAXBContext.newInstance(Person.class);<br> ObjectFactory factory = new ObjectFactory();</font></p> <p><font color="#0000ff" face="Courier New"> Person person = factory.createPerson();<br> person.setName("Matt Shannon");</font></p> <p><font color="#0000ff" face="Courier New"> GadgetMap<String, Gadget> map = new GadgetMap<String, Gadget>();</font></p> <p><font color="#0000ff" face="Courier New"> Gadget gadget1 = new Gadget();<br> gadget1.setMake("Apple");<br> gadget1.setModel("iPod");<br> gadget1.setYear(2002);</font></p> <p><font color="#0000ff" face="Courier New"> Computer gadget2 = new Computer();<br> gadget2.setMake("Lenovo");<br> gadget2.setModel("Thinkpad X230");<br> gadget2.setYear(2012);<br> gadget2.setCpu("Intel i5-3320M");<br> gadget2.setSpeed(2600);</font></p> <p><font color="#0000ff" face="Courier New"> map.put("my ipad", gadget1);<br> map.put("my laptop", gadget2);</font></p> <p><font color="#0000ff" face="Courier New"> person.setGadgets(map);</font></p> <p><font color="#0000ff" face="Courier New"> Marshaller marshaller = jc.createMarshaller();<br> marshaller.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, true);<br> marshaller.marshal(person, System.out);<br> }<br>}</font></p> <p><u>UnmarshalTest.java</u></p> <p><font color="#0000ff" face="Courier New">package com.blogspot.todayguesswhat.test.model;<br> <br>import java.io.File;</font></p> <p><font color="#0000ff" face="Courier New">import java.util.Map;</font></p> <p><font color="#0000ff" face="Courier New">import javax.xml.bind.JAXBContext;<br>import javax.xml.bind.Unmarshaller;<br> <br>public class UnmarshalTest<br>{<br> public static void main(String[] args) throws Exception<br> {<br> JAXBContext jc = JAXBContext.newInstance(Person.class);</font></p> <p><font color="#0000ff" face="Courier New"> Unmarshaller u = jc.createUnmarshaller();</font></p> <p><font color="#0000ff" face="Courier New"> Person person = (Person) u.unmarshal(new File(args[0]));<br> logPerson(person);<br> }</font></p> <p><font color="#0000ff" face="Courier New"> public static void logPerson(Person p)<br> {<br> if (p != null)<br> {<br> System.out.println("Person [name=" + p.getName() + "]");<br> GadgetMap<String, Gadget> map = p.getGadgets();<br> if (map != null)<br> {<br> for (Map.Entry<String,Gadget> entry : map.entrySet())<br> {<br> Gadget g = entry.getValue();<br> System.out.println(" " + entry.getKey() + " : " + gadgetToString(g));<br> }<br> }<br> }<br> }</font></p> <p><font color="#0000ff" face="Courier New"> public static String gadgetToString(Gadget g)<br> {<br> String result = null;<br> if (g != null)<br> {<br> result = "[make="+g.getMake()+"][model="+g.getModel()+"][year="+g.getYear()+"]";<br> if (g instanceof Computer)<br> {<br> Computer c = (Computer) g;<br> result = result + "[cpu="+c.getCpu()+"][speed="+c.getSpeed()+"]";<br> }<br> }<br> return result;<br> }<br>}</font></p> <p>An XML file leveraged by the unmarshalling test</p> <p><u>person.xml</u></p> <p><font color="#646b86" face="Courier New"><?xml version="1.0" encoding="UTF-8" standalone="yes"?><br><Person xmlns="</font><a href="http://todayguesswhat.blogspot.com/test"><font color="#646b86" face="Courier New">http://todayguesswhat.blogspot.com/test</font></a><font color="#646b86" face="Courier New">"><br> <name>Louise</name><br> <gadgets><br> <entry><br> <key>work laptop</key><br> <value xmlns:xsi="</font><a href="http://www.w3.org/2001/XMLSchema-instance"><font color="#646b86" face="Courier New">http://www.w3.org/2001/XMLSchema-instance</font></a><font color="#646b86" face="Courier New">" xsi:type="Computer"><br> <make>Lenovo</make><br> <model>Thinkpad X120e</model><br> <year>2011</year><br> <speed>1600</speed><br> <cpu>AMD E-350</cpu><br> </value><br> </entry><br> <entry><br> <key>my phone</key><br> <value><br> <make>Apple</make><br> <model>iPhone 3g</model><br> <year>2008</year><br> </value><br> </entry><br> </gadgets><br></Person></font></p> <p>And some windows bat files to invoke the various commands</p> <p><u>clean.bat</u></p> <p><font face="Courier New">@ECHO off<br>SET PROJECT_GENSRC=%~dp0\gensrc<br>IF EXIST "%PROJECT_GENSRC%" RMDIR /s /q "%PROJECT_GENSRC%"<br>SET PROJECT_JAVADOC=%~dp0\javadoc<br>IF EXIST "%PROJECT_JAVADOC%" RMDIR /s /q "%PROJECT_JAVADOC%"<br>SET PROJECT_CLASSES=%~dp0\classes<br>IF EXIST "%PROJECT_CLASSES%" RMDIR /s /q "%PROJECT_CLASSES%"<br>pause.</font></p> <p><u>jaxb.bat</u></p> <p><font face="Courier New">@ECHO off<br>SET JAVA_HOME=C:\Oracle\Middleware\jdk160_24<br>SET JAXBCMD=%JAVA_HOME%\bin\xjc.exe<br>SET PROJECT_GENSRC=%~dp0\gensrc<br>IF EXIST "%PROJECT_GENSRC%" RMDIR /s /q "%PROJECT_GENSRC%"<br>MKDIR "%PROJECT_GENSRC%"<br>"%JAXBCMD%" -no-header -d "%PROJECT_GENSRC%" -p com.blogspot.todayguesswhat.test.model -xmlschema "%~dp0\schema.xsd" -b "%~dp0\bindings.xml"<br>pause.</font></p> <p><a href="http://lh5.ggpht.com/-eju9gYkcWV0/UFNGTZMwLJI/AAAAAAAAAJE/OWVWcgmbi7w/s1600-h/image%25255B3%25255D.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh5.ggpht.com/-pEeyJot5S_o/UFNGUXtMfSI/AAAAAAAAAJM/dBKVTLd8Ddw/image_thumb%25255B1%25255D.png?imgmax=800" width="628" height="244"></a></p> <p><a href="http://lh6.ggpht.com/-rBFTWUiFBKg/UFNGVCQ6bpI/AAAAAAAAAJU/fIffLcHkqoA/s1600-h/image%25255B7%25255D.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh3.ggpht.com/-IoWXRNIDBVA/UFNGV_hcNRI/AAAAAAAAAJc/171ONWgsTPY/image_thumb%25255B3%25255D.png?imgmax=800" width="561" height="212"></a></p> <p><a href="http://lh5.ggpht.com/-3emkDVJILHk/UFNGWxFmtUI/AAAAAAAAAJk/xtXzC9UiXcQ/s1600-h/image%25255B12%25255D.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh4.ggpht.com/-lVJRQsinx44/UFNGYHS4hlI/AAAAAAAAAJs/ASY5HamRQ1c/image_thumb%25255B6%25255D.png?imgmax=800" width="605" height="267"></a></p> <p><u>compile.bat</u></p> <p><font face="Courier New">@ECHO off<br>SET JAVA_HOME=C:\Oracle\Middleware\jdk160_24<br>SET JAVACMPCMD=%JAVA_HOME%\bin\javac.exe<br>SET SRC_BASE_PKG=src\com\blogspot\todayguesswhat\test\model<br>SET GENSRC_BASE_PKG=gensrc\com\blogspot\todayguesswhat\test\model<br>SET PROJECT_CLASSES=%~dp0\classes<br>IF EXIST "%PROJECT_CLASSES%" RMDIR /s /q "%PROJECT_CLASSES%"<br>MKDIR "%PROJECT_CLASSES%"<br>"%JAVACMPCMD%" -d "%PROJECT_CLASSES%" %SRC_BASE_PKG%\*.java %GENSRC_BASE_PKG%\*.java<br>pause.</font></p> <p><a href="http://lh4.ggpht.com/-6oIKGUHbp-g/UFNGZIvMZzI/AAAAAAAAAJ0/ntU1L2zT0Pc/s1600-h/image%25255B16%25255D.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh6.ggpht.com/-XbFqvn4Pq74/UFNGa32HSYI/AAAAAAAAAJ8/ii4AUEYfiyg/image_thumb%25255B8%25255D.png?imgmax=800" width="606" height="257"></a></p> <p><u>javadoc.bat</u></p> <p><font face="Courier New">@ECHO off<br>SET JAVA_HOME=C:\Oracle\Middleware\jdk160_24<br>SET JAVADOCCMD=%JAVA_HOME%\bin\javadoc.exe<br>SET PROJECT_SRC=%~dp0\src<br>SET PROJECT_GENSRC=%~dp0\gensrc<br>SET PROJECT_JAVADOC=%~dp0\javadoc<br>IF EXIST "%PROJECT_JAVADOC%" RMDIR /s /q "%PROJECT_JAVADOC%"<br>MKDIR "%PROJECT_JAVADOC%"<br>"%JAVADOCCMD%" -sourcepath "%PROJECT_GENSRC%;%PROJECT_SRC%" -d "%PROJECT_JAVADOC%" -subpackages com.blogspot.todayguesswhat.test.model -protected<br>pause.</font></p> <p><a href="http://lh6.ggpht.com/-yCz-a1f4sz4/UFNGco1kNWI/AAAAAAAAAKA/4ItJ9wss0_Y/s1600-h/image%25255B20%25255D.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh6.ggpht.com/-irjC3rzwfdA/UFNGd49Dd3I/AAAAAAAAAKI/F7XRpB8YQQ0/image_thumb%25255B10%25255D.png?imgmax=800" width="606" height="284"></a></p> <p><a href="http://lh4.ggpht.com/--7LmJ-uV70k/UFNGeyDJKHI/AAAAAAAAAKU/qfrqduSd9lM/s1600-h/image%25255B24%25255D.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh4.ggpht.com/-RJDRGrwv-lQ/UFNGgoj65pI/AAAAAAAAAKc/UPFNjgKv_Rw/image_thumb%25255B12%25255D.png?imgmax=800" width="672" height="335"></a></p> <p> </p> <p><a href="http://lh4.ggpht.com/-Rxo_bYdSA5A/UFNGh1jx30I/AAAAAAAAAKg/f0qkmNOVfg4/s1600-h/image%25255B32%25255D.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh3.ggpht.com/-fNhwHAF5q80/UFNGi_L4p3I/AAAAAAAAAKs/lOMR_u1XYjI/image_thumb%25255B16%25255D.png?imgmax=800" width="866" height="319"></a></p> <p> </p> <p><a href="http://lh5.ggpht.com/-jWGimtlNTtw/UFNGknN_iRI/AAAAAAAAAK0/aG9B_e2VqXA/s1600-h/image%25255B39%25255D.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh6.ggpht.com/-lyUORWHnc50/UFNGlxw_VQI/AAAAAAAAAK8/2odUKl2NOa4/image_thumb%25255B19%25255D.png?imgmax=800" width="743" height="278"></a></p> <p> </p> <p><a href="http://lh4.ggpht.com/-r7gouAKihX0/UFNGmypd2BI/AAAAAAAAALE/9sW3fqWowTM/s1600-h/image%25255B40%25255D.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh4.ggpht.com/-zgtEySrf41A/UFNGoKYo0dI/AAAAAAAAALM/kdaFfGPrb2M/image_thumb%25255B20%25255D.png?imgmax=800" width="743" height="278"></a></p> <p><u>run-marshal-test.bat</u></p> <p><font face="Courier New">@ECHO off<br>MODE 120,50<br>SET JAVA_HOME=C:\Oracle\Middleware\jdk160_24<br>SET JAVACMD=%JAVA_HOME%\bin\java.exe<br>SET PROJECT_CLASSES=%~dp0\classes<br>"%JAVACMD%" -classpath "%PROJECT_CLASSES%" com.blogspot.todayguesswhat.test.model.MarshalTest<br>pause.</font></p> <p><a href="http://lh4.ggpht.com/-KpcsmiNoi4g/UFNGpEw0xiI/AAAAAAAAALU/cUSxfNXjX8E/s1600-h/image%25255B58%25255D.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh6.ggpht.com/-8GafAGIM_Dg/UFNGqU2l1ZI/AAAAAAAAALc/YVuc2zNHhxs/image_thumb%25255B30%25255D.png?imgmax=800" width="649" height="314"></a></p> <p><a href="http://lh5.ggpht.com/-IKlU9Pabbrg/UFNGrdTgP9I/AAAAAAAAALk/QGHEE-4cMUs/s1600-h/image%25255B59%25255D.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh5.ggpht.com/-UsR2DzdjaPc/UFNGsjRnBiI/AAAAAAAAALs/zGtxv1I-wUE/image_thumb%25255B31%25255D.png?imgmax=800" width="819" height="362"></a></p> <p><u>run-unmarshal-test.bat</u></p> <p><font face="Courier New">@ECHO off<br>MODE 120,50<br>SET JAVA_HOME=C:\Oracle\Middleware\jdk160_24<br>SET JAVACMD=%JAVA_HOME%\bin\java.exe<br>SET PROJECT_CLASSES=%~dp0\classes<br>"%JAVACMD%" -classpath "%PROJECT_CLASSES%" com.blogspot.todayguesswhat.test.model.UnmarshalTest "%~dp0\person.xml"<br>pause.</font></p> <p><a href="http://lh3.ggpht.com/-TLsFm5Tf_bo/UFNGtgoRy1I/AAAAAAAAAL0/0JA0r8UYKq4/s1600-h/image%25255B66%25255D.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh3.ggpht.com/-Lpwgwfum23E/UFNGvQ8OboI/AAAAAAAAAL8/xYFaQdYnX3k/image_thumb%25255B34%25255D.png?imgmax=800" width="639" height="310"></a></p> <p><a href="http://lh5.ggpht.com/-_e_QVQiGZuw/UFNGwAC97vI/AAAAAAAAAME/U8AQPrFGZUQ/s1600-h/image%25255B67%25255D.png"><img style="background-image: none; border-right-width: 0px; padding-left: 0px; padding-right: 0px; display: inline; border-top-width: 0px; border-bottom-width: 0px; border-left-width: 0px; padding-top: 0px" title="image" border="0" alt="image" src="http://lh6.ggpht.com/-KgNxUiukKnY/UFNGxSvtNjI/AAAAAAAAAMM/cgbYpzlVFnM/image_thumb%25255B35%25255D.png?imgmax=800" width="761" height="102"></a></p> <p><a href="https://docs.google.com/open?id=0BwNF4UK4Vp8bNzY3YlpMNjdCVTA" target="_blank">Click here to download a zip of this JAXB sample.</a></p> Matt (Brisbane/Australia)http://www.blogger.com/profile/16649081553551301779noreply@blogger.com0tag:blogger.com,1999:blog-2959498105239183642.post-51507592874060629222012-08-31T00:10:00.001+10:002013-05-20T23:58:17.232+10:00Windows 7 Replacement for UserAccounts.CommonDialog in VBScript<p>After 6 years of mostly trouble-free development/engineering, I finally retired my Windows XP-based Dell Latitude D620 from active work duty. I had been holding out for a business laptop with USB 3.0 to become available on the internal procurement site, and were finally able to obtain a Lenovo X230 with an Ivy Bridge i5-3320M processor. (Un)fortunately this new machine is running Windows 7 x64, and for that matter a bastardized version full of all the resource hungry corporate mandated bloat. This is the first time I have seen/used Windows 7 (having managed to also completely avoid Vista). The first thing I find myself doing is trying to make Windows 7 feel and behave like Windows XP again. After installing classic shell to get the old start button functionality back (<a href="http://classicshell.sourceforge.net/">http://classicshell.sourceforge.net/</a>) and turning off all of the visual effects (aka – ‘Adjust for best performance’ setting), my desktop is starting to resemble and feel like the ugly but reliable XP again. Now I’m slowly working through my kit bag of productivity scripts that I created for XP and trying to get these to function in Windows 7.</p> <p>One of the more frequent scripts I leverage is a simple VBScript for upload and download of a file by invoking the command-line FTP utility shipped with Windows. See the following article I wrote for the full original XP supported source code: <a href="http://todayguesswhat.blogspot.com.au/2010/06/vbscript-ftp-upload-sample-leverages.html">http://todayguesswhat.blogspot.com.au/2010/06/vbscript-ftp-upload-sample-leverages.html</a></p> <p>I found out that the <font color="#0000ff" size="2" face="Courier New">UserAccounts.CommonDialog<font color="#444444" face="Arial"> class/control is not available in Windows 7. I leveraged this control to allow the user to select a file for upload. Original VBScript code shown below:</font></font></p> <p><font color="#0000ff" size="2" face="Courier New">Function ChooseFile(initialDir) <br> Set cd = CreateObject("UserAccounts.CommonDialog") </font></p> <p><font color="#0000ff" size="2" face="Courier New"> cd.InitialDir = initialDir <br> cd.Filter = "ZIP files|*.zip|Text Documents|*.txt|Shell Scripts|*.*sh|All Files|*.*" <br> ' filter index 4 would show all files by default <br> ' filter index 1 would should zip files by default <br> cd.FilterIndex = 1 <br> If cd.ShowOpen = True Then <br> ChooseFile = cd.FileName <br> Else <br> ChooseFile = "" <br> End If <br> Set cd = Nothing <br>End Function </font></p><font color="#0000ff" size="2" face="Courier New"> <p><font color="#0000ff" size="2" face="Courier New"><font color="#444444" face="Arial">For Windows 7, I’ve kludged together code to replace the above method using techniques/articles/suggestions borrowed from multiple parties. If there is a cleaner mechanism to navigate and select a file using VBScript in Windows 7, please let me know :)</font></font></p> <p><font color="#444444" face="Arial">The code I developed/hacked-together creates a temporary powershell script that spawns System.Windows.Forms OpenFileDialog, and then writes the chosen file out to a temporary output text file. The VBScript then reads in the value from the output text file and returns that in the function. Code is as follows:</font></p> <p>Function ChooseFile (ByVal initialDir)</p> <p> Set shell = CreateObject("WScript.Shell")</p> <p> Set fso = CreateObject("Scripting.FileSystemObject")</p> <p> tempDir = shell.ExpandEnvironmentStrings("%TEMP%")</p> <p> tempFile = tempDir & "\" & fso.GetTempName</p> <p> ' temporary powershell script file to be invoked<br> powershellFile = tempFile & ".ps1"</p> <p> ' temporary file to store standard output from command<br> powershellOutputFile = tempFile & ".txt"</p> <p> 'input script<br> psScript = psScript & "[System.Reflection.Assembly]::LoadWithPartialName(""System.windows.forms"") | Out-Null" & vbCRLF<br> psScript = psScript & "$dlg = New-Object System.Windows.Forms.OpenFileDialog" & vbCRLF<br> psScript = psScript & "$dlg.initialDirectory = """ &initialDir & """" & vbCRLF<br> psScript = psScript & "$dlg.filter = ""ZIP files|*.zip|Text Documents|*.txt|Shell Scripts|*.*sh|All Files|*.*""" & vbCRLF<br> ' filter index 4 would show all files by default<br> ' filter index 1 would should zip files by default<br> psScript = psScript & "$dlg.FilterIndex = 4" & vbCRLF<br> psScript = psScript & "$dlg.Title = ""Select a file to upload""" & vbCRLF<br> psScript = psScript & "$dlg.ShowHelp = $True" & vbCRLF<br> psScript = psScript & "$dlg.ShowDialog() | Out-Null" & vbCRLF<br> psScript = psScript & "Set-Content """ &powershellOutputFile & """ $dlg.FileName" & vbCRLF<br> MsgBox psScript<br> <br> Set textFile = fso.CreateTextFile(powershellFile, True)<br> textFile.WriteLine(psScript)<br> textFile.Close<br> Set textFile = Nothing </p> <p> ' objShell.Run (strCommand, [intWindowStyle], [bWaitOnReturn]) <br> ' 0 Hide the window and activate another window.<br> ' bWaitOnReturn set to TRUE - indicating script should wait for the program <br> ' to finish executing before continuing to the next statement</p> <p> Dim appCmd<br> appCmd = "powershell -ExecutionPolicy unrestricted &'" & powershellFile & "'"<br> MsgBox appCmd<br> shell.Run appCmd, 0, TRUE</p> <p> ' open file for reading, do not create if missing, using system default format<br> Set textFile = fso.OpenTextFile(powershellOutputFile, 1, 0, -2)<br> ChooseFile = textFile.ReadLine<br> textFile.Close<br> Set textFile = Nothing<br> fso.DeleteFile(powershellFile)<br> fso.DeleteFile(powershellOutputFile)</p> <p>End Function</p> <p><strong><u><font face="Arial">UPDATE – May 2013</font></u></strong></p> <p><strong></strong><font face="Arial">Some commenters have suggested leveraging BrowseForFolder. At least for me, this produces strange behaviour on Windows 7 and may return -2147467259 (80004005) error code for certain file types (for example txt files) - but not others (e.g. zip). I would NOT recommend it.</font></p> <p><font face="Arial">Here is a a new and improved version which is must faster than above and should be backward compatible with XP:-</font></p> <p><font color="#333333">Set shell = CreateObject( "WScript.Shell" )<br>defaultLocalDir = shell.ExpandEnvironmentStrings("%USERPROFILE%") & "\Desktop"<br>Set shell = Nothing</font></p> <p><font color="#333333">file = ChooseFile(defaultLocalDir)<br>MsgBox file</font></p> <p><font color="#333333">Function ChooseFile (ByVal initialDir)<br> Set objWMIService = GetObject("winmgmts:\\.\root\cimv2")</font></p> <p><font color="#333333"> Set colItems = objWMIService.ExecQuery("Select * from Win32_OperatingSystem")<br> Dim winVersion</font></p> <p><font color="#333333"> ' This collection should contain just the one item<br> For Each objItem in colItems<br> 'Caption e.g. Microsoft Windows 7 Professional<br> 'Name e.g. Microsoft Windows 7 Professional |C:\windows|...<br> 'OSType e.g. 18 / OSArchitecture e.g 64-bit<br> 'Version e.g 6.1.7601 / BuildNumber e.g 7601<br> winVersion = CInt(Left(objItem.version, 1))<br> Next<br> Set objWMIService = Nothing<br> Set colItems = Nothing</font></p> <p><font color="#333333"> If (winVersion <= 5) Then<br> ' Then we are running XP and can use the original mechanism<br> Set cd = CreateObject("UserAccounts.CommonDialog")<br> cd.InitialDir = initialDir<br> cd.Filter = "ZIP files|*.zip|Text Documents|*.txt|Shell Scripts|*.*sh|All Files|*.*"<br> ' filter index 4 would show all files by default<br> ' filter index 1 would show zip files by default<br> cd.FilterIndex = 1<br> If cd.ShowOpen = True Then<br> ChooseFile = cd.FileName<br> Else<br> ChooseFile = ""<br> End If<br> Set cd = Nothing </font></p> <p><font color="#333333"> Else<br> ' We are running Windows 7 or later<br> Set shell = CreateObject( "WScript.Shell" )<br> Set ex = shell.Exec( "mshta.exe ""about: <input type=file id=X><script>X.click();new ActiveXObject('Scripting.FileSystemObject').GetStandardStream(1).WriteLine(X.value);close();resizeTo(0,0);</script>""" )<br> ChooseFile = Replace( ex.StdOut.ReadAll, vbCRLF, "" )</font></p> <p><font color="#333333"> Set ex = Nothing<br> Set shell = Nothing<br> End If<br>End Function <br></font><br></p></font> <p><font color="#0000ff" size="2" face="Courier New"></font></p> Matt (Brisbane/Australia)http://www.blogger.com/profile/16649081553551301779noreply@blogger.com9tag:blogger.com,1999:blog-2959498105239183642.post-26886784225570868462012-07-17T13:12:00.001+10:002012-07-18T00:00:43.938+10:00File Download Java Servlet example - 2GB overflow workaround<p>I discovered a few days back an issue with our product on HTTP downloads > 2GB.  It appears to be a simple overflow on the HttpServletResponse.setContentLength method. You can probably excuse the API designers circa 1997 assuming a 32-bit signed Integer with max value 2<sup>31</sup>-1 (2147483647 bytes) would be sufficient.  The Gigabit ethernet standard did not come for another year (1998)!</p> <p>Here is the exception seen when you provide a long value greater than 2147483647 bytes to the setContentLength(int) method:</p> <p>java.net.ProtocolException: Exceeded stated content-length of: '-XXXX' bytes <br />        at weblogic.servlet.internal.ServletOutputStreamImpl.checkCL(ServletOutputStreamImpl.java:200)</p> <p>Below is a sample download servlet with workaround for the 2gb limitation.  It has been tested on Firefox 3.6 against WebLogic Server 10.3.6 with a 2.2GB download and worked perfectly.</p> <p> <br /><font color="#800080" size="2" face="courier new">import java.io.FileInputStream; <br />import java.io.IOException; <br />import java.io.InputStream;</font></p> <p><font color="#800080" size="2" face="courier new">import javax.servlet.ServletContext; <br />import javax.servlet.ServletException; <br />import javax.servlet.ServletOutputStream;</font></p> <p><font color="#800080" size="2" face="courier new">import javax.servlet.http.HttpServlet; <br />import javax.servlet.http.HttpServletRequest; <br />import javax.servlet.http.HttpServletResponse;</font></p> <p> <br /><font color="#800080" size="2" face="courier new">public class DownloadServlet <br />  extends HttpServlet <br />{ <br />  @SuppressWarnings("compatibility:1533750721037291976") <br />  private static final long serialVersionUID = 1L;</font></p> <p><font color="#800080" size="2" face="courier new">  protected void doGet(HttpServletRequest request, <br />    HttpServletResponse response) <br />    throws ServletException, IOException <br />  { <br />    doPost(request, response); <br />  }</font></p> <p><font color="#800080" size="2" face="courier new">  protected void doPost(HttpServletRequest request, <br />    HttpServletResponse response) <br />    throws ServletException, IOException <br />  { <br />    // if no file parameter specified, download hosts file <br />    String file = request.getParameter("file"); <br />    file = (file == null || file.length() == 0) ? "/etc/hosts" : file;</font></p> <p><font color="#800080" size="2" face="courier new">    File fileObj = new File(file); <br />    if ((!fileObj.exists()) || (!fileObj.isFile()) || (!fileObj.canRead())) <br />    { <br />      throw new IOException("'file' '" + file + "' cannot be read."); <br />    }</font></p> <p><font color="#800080" size="2" face="courier new">    ServletContext context = getServletConfig().getServletContext();</font></p> <p><font color="#800080" size="2" face="courier new">    String mimetype = context.getMimeType(file); <br />    response.setContentType(mimetype == null ? "application/octet-stream" : <br />        mimetype);</font></p> <p><font color="#ff0000" size="2" face="Courier New"><strong>    long length = fileObj.length(); <br />    if (length <= Integer.MAX_VALUE) <br />    { <br />      response.setContentLength((int)length); <br />    } <br />    else <br />    { <br />      response.addHeader("Content-Length", Long.toString(length)); <br />    }</strong></font></p> <p><font color="#800080" size="2" face="courier new">    response.setHeader("Content-Disposition", <br />        "attachment; filename=\"" + fileObj.getName() + "\"");</font></p> <p><font color="#800080" size="2" face="courier new">    ServletOutputStream out = response.getOutputStream(); <br />    InputStream in = null; <br />    byte[] buffer = new byte[32768]; <br />    try <br />    { <br />      in = new FileInputStream(fileObj);</font></p> <p><font color="#800080" size="2" face="courier new">      int bytesRead; <br />      while ((bytesRead = in.read(buffer)) >= 0) <br />      { <br />        out.write(buffer, 0, bytesRead); <br />      } <br />    } <br />    finally <br />    { <br />      if (in != null) <br />      { <br />        in.close(); <br />      } <br />    } <br />  } <br />} <br /></font></p> Matt (Brisbane/Australia)http://www.blogger.com/profile/16649081553551301779noreply@blogger.com0tag:blogger.com,1999:blog-2959498105239183642.post-85755094962158778382012-07-07T17:52:00.001+10:002012-07-19T16:09:55.344+10:00WebLogic HTTPS One-Way SSL Tutorial<p>One-way SSL is the mode which most "storefronts" run on the internet so as to be able to accept credit card details and the like without the customer’s details being sent effectively in the clear from a packet-capture perspective.  In this mode, the server must present a valid public certificate to the client, but the client is not required to present a certificate to the server. </p> <p>Two-way SSL is far less common on the internet and is often leveraged for client certificate-based authentication.</p> <p>In terms of configuring WebLogic Server to support one-way SSL, it is just a matter of setting up an identity keystore containing a valid private key and associated public certificate signed by a certificate authority.  If the client has no knowledge/trust of the certificate authority that signed the server’s public certificate, the client must manually import/accept the CA’s public certificate.</p> <p>WebLogic in fact ships with two demo keystores for testing purposes containing X509 artifacts, namely <font face="courier new">DemoIdentity.jks</font> and <font face="Courier New">DemoTrust.jks</font> (both found in <font face="courier new">$WL_HOME/server/lib</font>). The latter is intended for Two-Way SSL purposes, though is also frequently supplied to java clients that may want to connect to the WebLogic instance due to it containing the <font face="Courier New">CN=CertGenCAB…</font> WebLogic demo CA public certificate that issued the certificate in the DemoIdentity keystore.</p> <p><font size="2" face="Courier New">keytool -list -v -keystore $WL_HOME/server/lib/DemoIdentity.jks -storepass DemoIdentityKeyStorePassPhrase</font></p> <p><font color="#800080" size="2" face="Courier New">Keystore type: JKS <br />Keystore type: JKS <br />Keystore provider: SUN</font></p> <p><font color="#800080" size="2" face="Courier New">Your keystore contains 1 entry</font></p> <p><font color="#800080" size="2" face="Courier New">Alias name: demoidentity <br />Creation date: Jun 30, 2012 <br />Entry type: PrivateKeyEntry <br />Certificate chain length: 1 <br />Certificate[1]: <br />Owner: CN=xxxxxxxxxx, OU=FOR TESTING ONLY, O=MyOrganization, L=MyTown, ST=MyState, C=US <br />Issuer: CN=CertGenCAB, OU=FOR TESTING ONLY, O=MyOrganization, L=MyTown, ST=MyState, C=US <br />Serial number: 75e1d5b02ee5b71fd3b1431f483f49f1 <br />Valid from: Fri Jun 29 06:45:36 PDT 2012 until: Wed Jun 30 06:45:36 PDT 2027 <br />Certificate fingerprints: <br />         MD5:  BB:49:16:85:4E:84:66:09:8A:68:83:A4:A7:06:AB:80 <br />         SHA1: 1F:67:9C:78:C7:64:B4:0B:2F:A7:AA:B2:69:70:9D:47:47:D5:A0:A8 <br />         Signature algorithm name: MD5withRSA <br />         Version: 1</font></p> <font color="#800080" size="2" face="Courier New"> <p>... <br /></p> </font> <p><font size="2" face="Courier New">keytool -list -keystore $WL_HOME/server/lib/DemoTrust.jks -storepass DemoTrustKeyStorePassPhrase</font></p> <p><font color="#800080" size="2" face="Courier New">Keystore type: JKS <br />Keystore provider: SUN</font></p> <p><font color="#800080" size="2" face="Courier New">Your keystore contains 4 entries</font></p> <p><font color="#800080" size="2" face="Courier New">certgenca, Mar 22, 2002, trustedCertEntry, <br />Certificate fingerprint (MD5): 8E:AB:55:50:A4:BC:06:F3:FE:C6:A9:72:1F:4F:D3:89 <br />wlsdemocanew2, Jan 24, 2003, trustedCertEntry, <br />Certificate fingerprint (MD5): 5B:10:D5:3C:C8:53:ED:75:43:58:BF:D5:E5:96:1A:CF <br />wlsdemocanew1, Jan 24, 2003, trustedCertEntry, <br />Certificate fingerprint (MD5): A1:17:A1:73:9B:70:21:B9:72:85:4D:83:01:69:C8:37 <br />wlscertgencab, Jan 24, 2003, trustedCertEntry, <br />Certificate fingerprint (MD5): A2:18:4C:E0:1C:AB:82:A7:65:86:86:03:D0:B3:D8:FE <br /></font></p> <p>The WebLogic demo CA public certificate that issued the certificate in the DemoIdentity keystore …</p> <p><font size="2" face="Courier New">keytool -printcert -file $WL_HOME/server/lib/CertGenCA.der</font></p> <p><font color="#800080" size="2" face="Courier New">Owner: CN=CertGenCAB, OU=FOR TESTING ONLY, O=MyOrganization, L=MyTown, ST=MyState, C=US <br />Issuer: CN=CertGenCAB, OU=FOR TESTING ONLY, O=MyOrganization, L=MyTown, ST=MyState, C=US <br />Serial number: 234b5559d1fa0f3ff5c82bdfed032a87 <br />Valid from: Thu Oct 24 08:54:45 PDT 2002 until: Tue Oct 25 08:54:45 PDT 2022 <br />Certificate fingerprints: <br />         MD5:  A2:18:4C:E0:1C:AB:82:A7:65:86:86:03:D0:B3:D8:FE <br />         SHA1: F8:5D:49:A4:12:54:78:C7:BA:42:A7:14:3E:06:F5:1E:A0:D4:C6:59 <br />         Signature algorithm name: MD5withRSA <br />         Version: 3</font></p> <p><font color="#800080" size="2" face="Courier New">...</font></p> <p>In order to leverage the above keystores, it is just a matter of connecting to the Administration Console, then expand Environment and select Servers. <br />Choose the server for which you want to configure the identity and trust keystores, and select Configuration > Keystores.  <br />You would then fill in the relevant fields (keystore fully qualified path, type (JKS), and keystore access password). e.g.</p> <p><font size="2">Identity store location: </font><font size="2"><font face="Courier New">/u01/app/oracle/product/Middleware/wlserver_10.3/server/lib/DemoIdentity.jks <br /></font>Identity store type: <font face="Courier New">JKS</font> <br />Identity store password: <font face="Courier New">DemoIdentityKeyStorePassPhrase</font></font></p> <p><font size="2">Trust store location: </font><font size="2"><font face="Courier New">/u01/app/oracle/product/Middleware/wlserver_10.3/server/lib/DemoTrust.jks <br /></font>Trust store type: <font face="Courier New">JKS</font> <br />Trust store password: <font face="Courier New">DemoTrustKeyStorePassPhrase</font></font></p> <p>Next, you need to enable the SSL Listen Port for the server:</p> <p>Home >Summary of Servers >XXX > General <br /><font size="2">SSL Listen Port: Enabled (Check) <br />SSL Listen Port: XXXX </font></p> <p>Finally, you need to tell WebLogic the alias and password in order to access the private key from the Identity Store:</p> <p>Home >Summary of Servers >XXX > SSL <br /><font size="2">Identity and Trust Locations: Keystores <br />Private Key Alias: <font face="Courier New">demoidentity</font> <br />Private Key Passphrase: <font face="Courier New">DemoIdentityPassPhrase</font></font></p> <p><u>Note - you are likely to find that the Demo Identity and Demo Trust keystores are preconfigured out of the box!</u></p> <hr /> <p>For production setups, you should be generating your own key of required strength, and get the associated public certificate signed by a “real” certificate authority.  A “real” certificate authority could be hosted locally given a private client base, or it could be one of the main internet players (verisign and co) if the instance was to be accessed by the general public.</p> <p>The steps below are a hybrid approach that leverage production techniques, but using the demo WebLogic certificate authority and openssl to perform the signing.</p> <p>First, lets create our identity keystore along with a new keypair ...</p> <p><font size="2" face="Courier New">KEYSTORE_FILE=$WL_HOME/server/lib/identity.jks <br />KEYSTORE_PASSWORD=welcome1 <br />KEY_ALIAS=identity <br />KEY_PASSWORD=welcome1 <br /> <br />keytool -genkeypair -v -keystore "$KEYSTORE_FILE" -storepass "$KEYSTORE_PASSWORD" -keyalg RSA -keysize 1024 -validity 365 -keypass "$KEY_PASSWORD" -alias "$KEY_ALIAS" -dname "CN=`hostname -f`" <br /></font></p> <p><font color="#800080" size="2" face="Courier New">Generating 1,024 bit RSA key pair and self-signed certificate (SHA1withRSA) with a validity of 365 days <br />        for: CN=xxxxxxxxxx.xx.xxxxxx.com <br />[Storing /u01/app/oracle/product/Middleware/wlserver_10.3/server/lib/identity.jks] <br /></font></p> <p>The above certificate is currently self-signed (such that, issuer = subject , private key signed its associated public certificate)</p> <p>For a production server we want to get our public certificate signed by a valid certificate authority (CA). <br />As we are just testing, we will use the WebLogic demo CA instead.</p> <p>We first need to get a certificate signing request ready ...</p> <p><font size="2" face="Courier New">KEYSTORE_FILE=$WL_HOME/server/lib/identity.jks <br />KEYSTORE_PASSWORD=welcome1 <br />KEY_ALIAS=identity <br />SIGNING_REQ_FILE="$WL_HOME/server/lib/identity_cert_signing_request.pem"</font></p> <p><font size="2" face="Courier New">keytool -certreq -v -keystore "$KEYSTORE_FILE" -storepass "$KEYSTORE_PASSWORD" -alias "$KEY_ALIAS" -file "$SIGNING_REQ_FILE"</font></p> <p><font color="#800080" size="2" face="Courier New">Certification request stored in file </u01/app/oracle/product/Middleware/wlserver_10.3/server/lib/identity_cert_signing_request.pem> <br />Submit this to your CA</font></p> <p><font size="2" face="Courier New">cat "$SIGNING_REQ_FILE"</font></p> <p><font color="#800080" size="2" face="Courier New">-----BEGIN NEW CERTIFICATE REQUEST----- <br />MIIBYjCBzAIBADAjMSEwHwYDVQQDExhhZGMyMTAwODY2LnVzLm9yYWNsZS5jb20wgZ8wDQYJKoZI <br />hvcNAQEBBQADgY0AMIGJAoGBAIOiOavPqXv3JDjYCsHyTmfprmSACDpF5dpozI2i2U8YVJaDYHtK <br />PYPdKu+KG7hpmxHlI6nHF2aqTid0sYqZjQDeYj16dHyLR6kh0SnAj3fA+I7ev9SFffyQ4cdpHwfV <br />1jfEkyPwNSQxz5X4UiRyd3bwCXI4rEPSjRUVpjPC8Q9LAgMBAAGgADANBgkqhkiG9w0BAQUFAAOB <br />gQAcieNA+waTBjDHRIrxDw+hSwjiB5OYaA6+smiEO8WYwNQkJifsr9OEc7SKeHmM2pAKGDUl9bKO <br />GHkv9FEKShNbAFXIRl0mItTwhIAkmqxKxZBv7HKmdM+lah5jpOQWmH1W4NRPbbGs/bdgYXHwEeYs <br />yCuaZsp90G73MxlPMgebFw== <br />-----END NEW CERTIFICATE REQUEST----- <br /></font></p> <p>Now we need to get the certificate signed.  As mentioned previously, we will use the WebLogic demo CA along with the openssl tool to generate the signed certificate.</p> <p>We first need to convert the WebLogic demo CA private key from DER to PEM format in order to use the openssl tool. <br />The private key as shown below is encrypted ....</p> <p><font size="2" face="Courier New">openssl asn1parse -inform DER -in CertGenCAKey.der</font></p> <p><font color="#800080" size="2" face="Courier New">    0:d=0  hl=4 l= 384 cons: SEQUENCE          <br />    4:d=1  hl=2 l=  26 cons: SEQUENCE          <br />    6:d=2  hl=2 l=   9 prim: OBJECT            :pbeWithMD5AndDES-CBC <br />   17:d=2  hl=2 l=  13 cons: SEQUENCE          <br />   19:d=3  hl=2 l=   8 prim: OCTET STRING      [HEX DUMP]:0011223344556677 <br />   29:d=3  hl=2 l=   1 prim: INTEGER           :05 <br />   32:d=1  hl=4 l= 352 prim: OCTET STRING      [HEX </font></p> <p><font color="#800080" size="2" face="Courier New">DUMP]:2A89DCA6697D9F...</font></p> <p><font color="#800080" size="2" face="Courier New">16</font></p> <p>The private key is protected by password based encryption - employing DES secret-key encryption in cipher-block chaining mode, where the secret key is derived from a password with the MD5 message-digest algorithm. </p> <p>The password used for encryption of this key is "password"</p> <p>Let's convert the key from DER format to PEM format supplying the encryption password …</p> <p><font size="2" face="Courier New">CA_KEY_DER="$WL_HOME/server/lib/CertGenCAKey.der" <br />CA_KEY_PEM="$WL_HOME/server/lib/CertGenCAKey.pem"</font></p> <p><font size="2" face="Courier New">openssl pkcs8 -inform DER -in "$CA_KEY_DER" -passin pass:password -outform PEM -out "$CA_KEY_PEM"</font></p> <p><font size="2" face="Courier New">cat "$CA_KEY_PEM"</font></p> <p><font size="2" face="Courier New"><font color="#800080">-----BEGIN RSA PRIVATE KEY----- <br />MIIBOgIBAAJBAK+kgUpzTgIJnC8rktjUQsYUy3X9JJP8OZokh0Pm2KH3bOK49CNI <br />/EmCFs1v2Gy2sOfWVP2HPY7uTEI1Yli3slsCAwEAAQJAM8oRxV4SVk93GgGDHyQX <br />PEjNZVDrBCeO5IB40fCzV64b33n7nCnrnemS8eJwvOs6+4dN4CpogCWKrMKkdRdS <br />gQIhAN56PT3B7/jS2D6HVgXATF+ThJ2HkbL7CDMPVkJE00sLAiEAyhussDRxqESf <br />w+EibP34Effs6LJP73BeI43yIgq6R/ECIF75HfWkOdYl5AxlZ9KSscfHSMCa5Bbc <br />1TY+4NrvWKfbAiEAmnpEp/mt1dt98g4mbgBAmBtT7Rg9mNneWnaoV/SzJUECICjQ <br />Lsm4mCSPorIOlGsk2TQPH0MzJsk4TziNnsB08jBO <br />-----END RSA PRIVATE KEY-----</font></font></p> <p> <br />We also need the CA’s public certificate in PEM format …</p> <p><font size="2" face="Courier New">CA_CERT_DER="$WL_HOME/server/lib/CertGenCA.der" <br />CA_CERT_PEM="$WL_HOME/server/lib/CertGenCA.pem"</font></p> <p><font size="2" face="Courier New">openssl x509 -inform DER -in "$CA_CERT_DER" -outform PEM -out "$CA_CERT_PEM"</font></p> <p><font size="2" face="Courier New">cat "$CA_CERT_PEM"</font></p> <p><font color="#800080" size="2" face="Courier New">-----BEGIN CERTIFICATE----- <br />MIICGDCCAcKgAwIBAgIQI0tVWdH6Dz/1yCvf7QMqhzANBgkqhkiG9w0BAQQFADB5 <br />MQswCQYDVQQGEwJVUzEQMA4GA1UECBMHTXlTdGF0ZTEPMA0GA1UEBxMGTXlUb3du <br />MRcwFQYDVQQKEw5NeU9yZ2FuaXphdGlvbjEZMBcGA1UECxMQRk9SIFRFU1RJTkcg <br />T05MWTETMBEGA1UEAxMKQ2VydEdlbkNBQjAeFw0wMjEwMjQxNTU0NDVaFw0yMjEw <br />MjUxNTU0NDVaMHkxCzAJBgNVBAYTAlVTMRAwDgYDVQQIEwdNeVN0YXRlMQ8wDQYD <br />VQQHEwZNeVRvd24xFzAVBgNVBAoTDk15T3JnYW5pemF0aW9uMRkwFwYDVQQLExBG <br />T1IgVEVTVElORyBPTkxZMRMwEQYDVQQDEwpDZXJ0R2VuQ0FCMFwwDQYJKoZIhvcN <br />AQEBBQADSwAwSAJBAK+kgUpzTgIJnC8rktjUQsYUy3X9JJP8OZokh0Pm2KH3bOK4 <br />9CNI/EmCFs1v2Gy2sOfWVP2HPY7uTEI1Yli3slsCAwEAAaMmMCQwDgYDVR0PAQH/ <br />BAQDAgIEMBIGA1UdEwEB/wQIMAYBAf8CAQEwDQYJKoZIhvcNAQEEBQADQQBCOCsQ <br />9QWvGvAikjBBOjrTFqZBNJYJsv5+mVZ90pVwmzEDihw8dbI3ubMYz5saXkKSy3rc <br />SvJK9nZJkHvb88hZ <br />-----END CERTIFICATE-----</font></p> <p>Once we have our CA key and certificate in PEM format, lets do the certificate signing ...</p> <p><font size="2" face="Courier New">CA_KEY_PEM="$WL_HOME/server/lib/CertGenCAKey.pem" <br />CA_CERT_PEM="$WL_HOME/server/lib/CertGenCA.pem" <br />CA_SERIAL_FILE="$WL_HOME/server/lib/CA.srl" <br />SIGNING_REQ_FILE="$WL_HOME/server/lib/identity_cert_signing_request.pem" <br />SIGNED_CERT="$WL_HOME/server/lib/identity_cert_signed.pem" <br /></font></p> <p><font size="2" face="Courier New">openssl x509 -req -days 365 -in "$SIGNING_REQ_FILE" -inform PEM -CA "$CA_CERT_PEM" -CAkey "$CA_KEY_PEM" -out "$SIGNED_CERT" -outform PEM -CAcreateserial -CAserial "$CA_SERIAL_FILE"</font></p> <p><font color="#800080" size="2" face="Courier New">Signature ok <br />subject=/CN=xxxxxxxxxx.xx.xxxxxx.com <br />Getting CA Private Key</font><font size="2" face="Courier New"></font> </p> <p><font size="2" face="Courier New">keytool -printcert -file "$SIGNED_CERT"</font></p> <p></p> <p><font size="2" face="Courier New"><font color="#800080">Owner: CN=xxxxxxxxxx.xx.xxxxxx.com <br />Issuer: CN=CertGenCAB, OU=FOR TESTING ONLY, O=MyOrganization, L=MyTown, ST=MyState, C=US <br />Serial number: d80dbb0b65dcc3b8 <br />Valid from: Fri Jul 06 20:41:08 PDT 2012 until: Sat Jul 06 20:41:08 PDT 2013 <br />Certificate fingerprints: <br />         MD5:  2E:2A:00:AD:A9:38:E9:FA:B7:C6:A8:85:33:74:72:B3 <br />         SHA1: 27:0D:9C:A9:39:37:92:E3:68:A2:61:E8:65:D3:3F:B9:D1:86:A8:E2 <br />         Signature algorithm name: SHA1withRSA <br />         Version: 1</font></font></p> <p>As can be seen above, the issuer and owner are now different.  The certificate is no longer self-signed, but rather signed by our CA. <br />Now we are ready to imported the signed public certificate back in to our identity keystore ...</p> <p><font size="2" face="Courier New">KEYSTORE_FILE=$WL_HOME/server/lib/identity.jks <br />KEYSTORE_PASSWORD=welcome1 <br />KEY_ALIAS=identity <br />SIGNED_CERT="$WL_HOME/server/lib/identity_cert_signed.pem"</font></p> <p><font size="2" face="Courier New">keytool -import -v -keystore "$KEYSTORE_FILE" -storepass "$KEYSTORE_PASSWORD" -alias "$KEY_ALIAS" -file "$SIGNED_CERT"</font></p> <p>You should see the following error when you perform the above command:</p> <p><font color="#800080" size="2" face="Courier New">keytool error: java.lang.Exception: Failed to establish chain from reply </font></p> <p>keytool prevents us from importing a certificate should it not be able to verify the full signing chain. As we leveraged the WebLogic demo CA, we need to import the CA's public certificate in to our keystore as a trusted certificate authority prior to importing our signed certificate.</p> <p><font size="2" face="Courier New">KEYSTORE_FILE=$WL_HOME/server/lib/identity.jks <br />KEYSTORE_PASSWORD=welcome1 <br />CA_ALIAS=wlsdemoca <br />CA_CERT_PEM="$WL_HOME/server/lib/CertGenCA.pem"</font></p> <p><font size="2" face="Courier New">keytool -import -v -keystore "$KEYSTORE_FILE" -storepass "$KEYSTORE_PASSWORD" -trustcacerts -alias "$CA_ALIAS" -file "$CA_CERT_PEM"</font></p> <p><font color="#800080" size="2" face="Courier New">Owner: CN=CertGenCAB, OU=FOR TESTING ONLY, O=MyOrganization, L=MyTown, ST=MyState, C=US <br />Issuer: CN=CertGenCAB, OU=FOR TESTING ONLY, O=MyOrganization, L=MyTown, ST=MyState, C=US <br />Serial number: 234b5559d1fa0f3ff5c82bdfed032a87 <br />Valid from: Thu Oct 24 08:54:45 PDT 2002 until: Tue Oct 25 08:54:45 PDT 2022 <br />Certificate fingerprints: <br />         MD5:  A2:18:4C:E0:1C:AB:82:A7:65:86:86:03:D0:B3:D8:FE <br />         SHA1: F8:5D:49:A4:12:54:78:C7:BA:42:A7:14:3E:06:F5:1E:A0:D4:C6:59 <br />         Signature algorithm name: MD5withRSA <br />         Version: 3</font></p> <p><font color="#800080" size="2" face="Courier New">...</font></p> <p><font color="#800080" size="2" face="Courier New">Trust this certificate? [no]:  y <br />Certificate was added to keystore <br />[Storing /u01/app/oracle/product/Middleware/wlserver_10.3/server/lib/identity.jks]</font></p> <p>Add the "<font face="courier ">-noprompt</font>" option to prevent the “Trust this certificate” prompt.</p> <p>Now lets retry our request to import the signed certificate ...</p> <p><font size="2" face="Courier New">KEYSTORE_FILE=$WL_HOME/server/lib/identity.jks <br />KEYSTORE_PASSWORD=welcome1 <br />KEY_ALIAS=identity <br />SIGNED_CERT="$WL_HOME/server/lib/identity_cert_signed.pem"</font></p> <p><font size="2" face="Courier New">keytool -import -v -keystore "$KEYSTORE_FILE" -storepass "$KEYSTORE_PASSWORD" -alias "$KEY_ALIAS" -file "$SIGNED_CERT" -noprompt</font></p> <p><font color="#800080" size="2" face="Courier New">Certificate reply was installed in keystore <br />[Storing /u01/app/oracle/product/Middleware/wlserver_10.3/server/lib/identity.jks]</font></p> <p>Let's verify the keystore contents ...</p> <p><font size="2" face="Courier New">KEYSTORE_FILE=$WL_HOME/server/lib/identity.jks <br />KEYSTORE_PASSWORD=welcome1</font></p> <p><font size="2" face="Courier New">keytool -list -v -keystore "$KEYSTORE_FILE" -storepass "$KEYSTORE_PASSWORD"</font></p> <p><font color="#800080" size="2" face="Courier New">Keystore type: JKS <br />Keystore provider: SUN</font></p> <p><font color="#800080" size="2" face="Courier New">Your keystore contains 2 entries</font></p> <p><font color="#800080" size="2" face="Courier New">Alias name: identity <br />Creation date: Jul 6, 2012 <br />Entry type: PrivateKeyEntry <br />Certificate chain length: 2 <br />Certificate[1]: <br />Owner: CN=xxxxxxxxxx.xx.xxxxxx.com <br />Issuer: CN=CertGenCAB, OU=FOR TESTING ONLY, O=MyOrganization, L=MyTown, ST=MyState, C=US <br />Serial number: d80dbb0b65dcc3b8 <br />Valid from: Fri Jul 06 20:41:08 PDT 2012 until: Sat Jul 06 20:41:08 PDT 2013 <br />Certificate fingerprints: <br />         MD5:  2E:2A:00:AD:A9:38:E9:FA:B7:C6:A8:85:33:74:72:B3 <br />         SHA1: 27:0D:9C:A9:39:37:92:E3:68:A2:61:E8:65:D3:3F:B9:D1:86:A8:E2 <br />         Signature algorithm name: SHA1withRSA <br />         Version: 1 <br />Certificate[2]: <br />Owner: CN=CertGenCAB, OU=FOR TESTING ONLY, O=MyOrganization, L=MyTown, ST=MyState, C=US <br />Issuer: CN=CertGenCAB, OU=FOR TESTING ONLY, O=MyOrganization, L=MyTown, ST=MyState, C=US <br />Serial number: 234b5559d1fa0f3ff5c82bdfed032a87 <br />Valid from: Thu Oct 24 08:54:45 PDT 2002 until: Tue Oct 25 08:54:45 PDT 2022 <br />Certificate fingerprints: <br />         MD5:  A2:18:4C:E0:1C:AB:82:A7:65:86:86:03:D0:B3:D8:FE <br />         SHA1: F8:5D:49:A4:12:54:78:C7:BA:42:A7:14:3E:06:F5:1E:A0:D4:C6:59 <br />         Signature algorithm name: MD5withRSA <br />         Version: 3</font></p> <p><font color="#800080" size="2" face="Courier New">...</font></p> <p><font color="#800080" size="2" face="Courier New">******************************************* <br />*******************************************</font></p> <p><font color="#800080" size="2" face="Courier New">Alias name: wlsdemoca <br />Creation date: Jul 6, 2012 <br />Entry type: trustedCertEntry</font></p> <p><font color="#800080" size="2" face="Courier New">Owner: CN=CertGenCAB, OU=FOR TESTING ONLY, O=MyOrganization, L=MyTown, ST=MyState, C=US <br />Issuer: CN=CertGenCAB, OU=FOR TESTING ONLY, O=MyOrganization, L=MyTown, ST=MyState, C=US <br />Serial number: 234b5559d1fa0f3ff5c82bdfed032a87 <br />Valid from: Thu Oct 24 08:54:45 PDT 2002 until: Tue Oct 25 08:54:45 PDT 2022 <br />Certificate fingerprints: <br />         MD5:  A2:18:4C:E0:1C:AB:82:A7:65:86:86:03:D0:B3:D8:FE <br />         SHA1: F8:5D:49:A4:12:54:78:C7:BA:42:A7:14:3E:06:F5:1E:A0:D4:C6:59 <br />         Signature algorithm name: MD5withRSA <br />         Version: 3</font></p> <p><font color="#800080" size="2" face="Courier New">...</font> <br /></p> <p>Next we go to the Administration Console and ensure for our Server that:</p> <ol> <li>“Custom Identity and Java Standard Trust” keystore configuration option is chosen </li> <li>The custom identity keystore path, type, and password is correctly set </li> <li>Java standard trust store password is set (“changeit”) </li> <li>The private key alias and password is set </li> <li>The SSL listen port is set </li> </ol> <p><font size="2">Identity store location: <font face="Courier New">/u01/app/oracle/product/Middleware/wlserver_10.3/server/lib/identity.jks <br /></font>Identity store type: <font face="Courier New">JKS</font> <br />Identity store password: <font face="Courier New">welcome1</font></font></p> <p><font size="2">Identity and Trust Locations: Keystores <br />Private Key Alias: <font face="Courier New">identity</font> <br />Private Key Passphrase: <font face="Courier New">welcome1</font></font></p> <p>As far as the trust keystore is concerned (and given we are leveraging one-way SSL), we just set the instance to utilize the standard trust keystore supplied with the JDK (<font face="Courier New">$JAVA_HOME/jre/lib/security/cacerts</font>   - password “<font face="Courier New">changeit</font>”).</p> <p></p> <hr /> <p></p> <p>Clients that will be connecting to our SSL protected server will likely have to import our CA’s public certificate in to their local trust keystore (cacerts etc) given that we are using a demo CA with a certificate chain unlikely to be known/resolvable.</p> <p>Failure to do this from a Java client will likely lead to an exception such as:</p> <p><font color="#ff0000" face="Courier New">javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: <br />  PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target</font></p> <p>As an alternative to using the JDK’s standard trust keystore, another option is create a separate keystore with just the CA’s public certificate chain.  You then provide explicit SSL trust store system properties on the java command line:</p> <p><font size="2" face="Courier New">KEYSTORE_FILE=$WL_HOME/server/lib/trust.jks <br />KEYSTORE_PASSWORD=welcome1 <br />CA_ALIAS=wlsdemoca <br />CA_CERT_PEM="$WL_HOME/server/lib/CertGenCA.pem"</font></p> <p><font size="2" face="Courier New">keytool -import -v -keystore "$KEYSTORE_FILE" -storepass "$KEYSTORE_PASSWORD" -trustcacerts -alias "$CA_ALIAS" -file "$CA_CERT_PEM" -noprompt</font></p> <p><font color="#800080" size="2" face="Courier New">Certificate was added to keystore <br />[Storing /u01/app/oracle/product/Middleware/wlserver_10.3/server/lib/trust.jks] <br /></font></p> <p>Let's verify the keystore contents ...</p> <p><font size="2" face="Courier New">KEYSTORE_FILE=$WL_HOME/server/lib/trust.jks <br />KEYSTORE_PASSWORD=welcome1 <br />keytool -list -v -keystore "$KEYSTORE_FILE" -storepass "$KEYSTORE_PASSWORD"</font></p> <p><font color="#800080" size="2" face="Courier New">Keystore type: JKS <br />Keystore provider: SUN</font></p> <p><font color="#800080" size="2" face="Courier New">Your keystore contains 1 entry</font></p> <p><font color="#800080" size="2" face="Courier New">Alias name: wlsdemoca <br />Creation date: Jul 9, 2012 <br />Entry type: trustedCertEntry</font></p> <p><font color="#800080" size="2" face="Courier New">Owner: CN=CertGenCAB, OU=FOR TESTING ONLY, O=MyOrganization, L=MyTown, ST=MyState, C=US <br />Issuer: CN=CertGenCAB, OU=FOR TESTING ONLY, O=MyOrganization, L=MyTown, ST=MyState, C=US <br />Serial number: 234b5559d1fa0f3ff5c82bdfed032a87 <br />Valid from: Thu Oct 24 08:54:45 PDT 2002 until: Tue Oct 25 08:54:45 PDT 2022 <br />Certificate fingerprints: <br />         MD5:  A2:18:4C:E0:1C:AB:82:A7:65:86:86:03:D0:B3:D8:FE <br />         SHA1: F8:5D:49:A4:12:54:78:C7:BA:42:A7:14:3E:06:F5:1E:A0:D4:C6:59 <br />         Signature algorithm name: MD5withRSA <br />         Version: 3</font></p> <p><font color="#800080" size="2" face="Courier New">...</font><font color="#800080" size="2" face="Courier New"></font></p> <p>Invoke Java with explicit SSL trust store elements set ...</p> <p><font size="2" face="Courier New">SSL_TRUST_STORE="-Djavax.net.ssl.trustStore=$WL_HOME/server/lib/trust.jks" <br />SSL_TRUST_PASSWORD="-Djavax.net.ssl.trustStorePassword=welcome1"</font></p> <p><font size="2" face="Courier New">$JAVA_HOME/bin/java "${SSL_TRUST_STORE}" "${SSL_TRUST_PASSWORD}" -cp ... XXX</font></p> Matt (Brisbane/Australia)http://www.blogger.com/profile/16649081553551301779noreply@blogger.com0tag:blogger.com,1999:blog-2959498105239183642.post-10859842200106254732012-06-27T16:47:00.001+10:002013-05-02T09:36:33.957+10:00Decrypt / Dump contents of CWALLET.SSO (Oracle file based credential store)<p>When using a file-based credential store with Oracle, credentials ultimately get stored in a wallet file (cwallet.sso)</p> <p>Very little if any info exists on how to dump the contents of the wallet. At best, most people leverage the trusty <font size="2" face="courier new">orapki</font> command to get an overview of what’s inside as far as the maps and keys, but actual password information is never divulged.</p> <p>For example:</p> <p><font color="#000080" size="2" face="Courier New"><font color="#800000">$MW_HOME/oracle_common/bin/orapki wallet display -wallet ~/cwallet.sso <br></font>Oracle PKI Tool : Version 11.1.1.6.0 <br>Copyright (c) 2004, 2011, Oracle and/or its affiliates. All rights reserved.</font></p> <p><font color="#000080" size="2" face="courier new">Requested Certificates: <br>User Certificates: <br>Oracle Secret Store entries: <br></font><a href="mailto:dip@#3#@cn=odisrv"><font color="#000080" size="2" face="courier new">dip@#3#@cn=odisrv</font></a> <br><a href="mailto:ODSMMap@#3#@ODSMKey.Wallet"><font color="#000080" size="2" face="courier new">ODSMMap@#3#@ODSMKey.Wallet</font></a> <br><a href="mailto:oracle.wsm.security@#3#@enc-csf-key"><font color="#000080" size="2" face="courier new"><strong>oracle.wsm.security@#3#@enc-csf-key</strong></font></a> <br><a href="mailto:oracle.wsm.security@#3#@keystore-csf-key"><font color="#000080" size="2" face="courier new"><strong>oracle.wsm.security@#3#@keystore-csf-key</strong></font></a> <br><a href="mailto:oracle.wsm.security@#3#@sign-csf-key"><font color="#000080" size="2" face="courier new"><strong>oracle.wsm.security@#3#@sign-csf-key</strong></font></a> <br><font color="#000080" size="2" face="courier new">Trusted Certificates: <br>Subject: OU=Class 1 Public Primary Certification Authority,O=VeriSign\, Inc.,C=US <br>Subject: OU=Secure Server Certification Authority,O=RSA Data Security\, Inc.,C=US <br>Subject: CN=Entrust.net Secure Server Certification Authority,OU=(c) 1999 Entrust.net Limited,OU=www.entrust.net/CPS incorp. by ref. (limits liab.),O=Entrust.net,C=US <br>Subject: CN=GTE CyberTrust Global Root,OU=GTE CyberTrust Solutions\, Inc.,O=GTE Corporation,C=US <br>Subject: OU=Class 3 Public Primary Certification Authority,O=VeriSign\, Inc.,C=US <br>Subject: CN=Entrust.net Secure Server Certification Authority,OU=(c) 2000 Entrust.net Limited,OU=www.entrust.net/SSL_CPS incorp. by ref. (limits liab.),O=Entrust.net <br>Subject: OU=Class 2 Public Primary Certification Authority,O=VeriSign\, Inc.,C=US <br>Subject: CN=Entrust.net Certification Authority (2048),OU=(c) 1999 Entrust.net Limited,OU=www.entrust.net/CPS_2048 incorp. by ref. (limits liab.),O=Entrust.net</font></p> <p>Until now that is :)</p> <p>Note - wallet and jps-config-dump file must reside in same location for this sample code to function!! In the example below, choose either the domain wallet, or the bootstrap wallet.</p> <p><font size="2" face="Courier New">PATH_TO_WALLET="$DOMAIN_HOME/config/fmwconfig/bootstrap/cwallet.sso"<br>PATH_TO_JPS="$DOMAIN_HOME/config/fmwconfig/bootstrap/jps-config-dump.xml"</font></p><font size="2" face="Courier New"> <p><font size="2" face="Arial">or …</font></p> <p><font size="2" face="Courier New">PATH_TO_WALLET="$DOMAIN_HOME/config/fmwconfig/cwallet.sso"<br>PATH_TO_JPS="$DOMAIN_HOME/config/fmwconfig/jps-config-dump.xml"</p></font></font> <p><font color="#800000" size="2" face="Courier New">cat > "${PATH_TO_JPS}" <<EOF <br><?xml version="1.0" encoding="UTF-8" standalone='yes'?> <br><jpsConfig xmlns="http://xmlns.oracle.com/oracleas/schema/11/jps-config-11_1.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://xmlns.oracle.com/oracleas/schema/11/jps-config-11_1.xsd</font><font color="#800000" size="2" face="Courier New"> jps-config-11_1.xsd" schema-major-version="11" schema-minor-version="1"> <br> <serviceProviders> <br> <serviceProvider type="CREDENTIAL_STORE" name="credstoressp" class="oracle.security.jps.internal.credstore.ssp.SspCredentialStoreProvider"> <br> <description>Credential Store Service Provider</description> <br> </serviceProvider> <br> </serviceProviders> <br> <serviceInstances> <br> <serviceInstance provider="credstoressp" name="credstore"> <br> <property value="file:${PATH_TO_WALLET}" name="location"/> <br> </serviceInstance> <br> </serviceInstances> <br> <jpsContexts default="test"> <br> <jpsContext name="test"> <br> <serviceInstanceRef ref="credstore"/> <br> </jpsContext> <br> </jpsContexts> <br></jpsConfig> <br>EOF</font></p> <p><font color="#000080" size="2" face="Courier New">cat > /tmp/DumpWallet.java <<EOF <br>import java.io.File;</font></p> <p><font color="#000080" size="2" face="Courier New">import java.util.Hashtable;</font></p> <p><font color="#000080" size="2" face="Courier New">import oracle.security.jps.JpsContext; <br>import oracle.security.jps.JpsContextFactory;</font></p> <p><font color="#000080" size="2" face="Courier New">import oracle.security.jps.service.credstore.Credential; <br>import oracle.security.jps.service.credstore.CredentialFactory; <br>import oracle.security.jps.service.credstore.CredentialMap; <br>import oracle.security.jps.service.credstore.CredentialStore; <br>import oracle.security.jps.service.credstore.GenericCredential; <br>import oracle.security.jps.service.credstore.PasswordCredential;</font></p> <p><font color="#000080" size="2" face="Courier New">public class DumpWallet <br>{ <br> private static final byte[] HEX = new byte[] { <br> '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'A', 'B', 'C', 'D', 'E', 'F' }; <br> <br> public static void main(String[] args) <br> { <br> try <br> { <br> System.setProperty("oracle.security.jps.config", (args.length > 0) ? args[0] : "jps-config.xml"); <br> // System.setProperty("java.security.debug", "all"); <br> JpsContextFactory ctxFactory = JpsContextFactory.getContextFactory(); <br> JpsContext ctx = ctxFactory.getContext(); <br> CredentialStore store = ctx.getServiceInstance(CredentialStore.class); <br> listAll(store); <br> } <br> catch (Exception e) <br> { <br> e.printStackTrace(); <br> } <br> } <br> <br> private static void listAll(CredentialStore store) throws Exception <br> { <br> System.out.println("Dumping store contents ..."); <br> for (String map : store.getMapNames()) <br> { <br> System.out.println("\n" + "### Map: " + map); <br> CredentialMap credMap = store.getCredentialMap(map); <br> if (credMap != null) <br> { <br> int i = 1; <br> for (String key : credMap.keySet()) <br> { <br> System.out.println(" " + i++ + ". + Key: " + key); <br> <br> Credential cred = credMap.getCredential(key); <br> System.out.println(" class = " + cred.getClass().getName()); <br> System.out.println(" desc = " + cred.getDescription());</font></p> <p><font color="#000080" size="2" face="Courier New"> if (cred instanceof PasswordCredential) <br> { <br> PasswordCredential pc = (PasswordCredential)cred; <br> System.out.println(" name = " + pc.getName()); <br> System.out.println(" pass = " + new String(pc.getPassword())); <br> System.out.println(" expires = " + pc.getExpiryTime()); <br> } <br> else if (cred instanceof GenericCredential) <br> { <br> GenericCredential gc = (GenericCredential)cred; <br> Object c = gc.getCredential(); <br> String type = (! c.getClass().isArray()) <br> ? c.getClass().getName() <br> : ("Array of " + c.getClass().getComponentType().getName());</font></p> <p><font color="#000080" size="2" face="Courier New"> System.out.println(" type = " + type); <br> if (c instanceof String) <br> { <br> System.out.println(" cred = " + c); <br> } <br> else if ( c instanceof Hashtable) <br> { <br> Hashtable ht = (Hashtable)c; <br> for (Object htkey : ht.keySet()) <br> { <br> Object htVal = ht.get(htkey); <br> if (htVal instanceof char[]) <br> { <br> System.out.println(" cred = (" + htkey + ", " + new String((char[])htVal) + ")"); <br> } <br> else <br> { <br> System.out.println(" cred = (" + htkey + ", " + htVal + ")"); <br> } <br> } <br> } <br> else if (c instanceof javax.crypto.spec.SecretKeySpec) <br> { <br> javax.crypto.spec.SecretKeySpec secret = (javax.crypto.spec.SecretKeySpec) c; <br> System.out.println(" algorith = " + secret.getAlgorithm()); <br> System.out.println(" format = " + secret.getFormat()); <br> System.out.println(" key material as hex = " + bytesAsHex(secret.getEncoded())); <br> } <br> else if (c instanceof byte[]) <br> { <br> System.out.println(" byte array as hex = " + bytesAsHex((byte[])c)); <br> } <br> System.out.println(" expires = " + gc.getExpiryTime()); <br> } <br> else <br> { <br> System.out.println(" toStr = " + cred.toString()); <br> } <br> } <br> } <br> } <br> }</font></p> <p><font color="#000080" size="2" face="Courier New"> public static final String bytesAsHex(byte[] bytes) <br> { <br> StringBuffer sb = new StringBuffer(); <br> for (int i = 0; i < bytes.length; i++) <br> { <br> sb.append((char)(HEX[(bytes[i] & 0x00F0) >> 4])).append((char)(HEX[bytes[i] & 0x000F])).append(" "); <br> } <br> return sb.toString(); <br> } <br>} <br>EOF</font></p> <p><font size="2" face="Courier New">CP=/tmp <br>CP=$CP:$MW_HOME/oracle_common/modules/oracle.jps_11.1.1/jps-api.jar <br>CP=$CP:$MW_HOME/oracle_common/modules/oracle.jps_11.1.1/jps-common.jar <br>CP=$CP:$MW_HOME/oracle_common/modules/oracle.jps_11.1.1/jps-internal.jar <br>CP=$CP:$MW_HOME/oracle_common/modules/oracle.idm_11.1.1/identitystore.jar <br>CP=$CP:$MW_HOME/oracle_common/modules/oracle.osdt_11.1.1/osdt_xmlsec.jar <br>CP=$CP:$MW_HOME/oracle_common/modules/oracle.pki_11.1.1/oraclepki.jar</font></p> <p><font size="2" face="Courier New">$JAVA_HOME/bin/javac -cp $CP /tmp/DumpWallet.java</font></p> <p><font size="2" face="Courier New">$JAVA_HOME/bin/java -cp $CP DumpWallet "${PATH_TO_JPS}"</font></p> <p>Sample Output :-</p> <p><font color="#800080" face="Courier New">Dumping store contents ...</font></p> <p><font color="#800080" face="Courier New">### Map: oracle.wsm.security <br>1. + Key: sign-csf-key <br> class = oracle.security.jps.internal.credstore.PasswordCredentialImpl <br> desc = signing key alias/password <br> name = orakey <br> pass = welcome1 <br> expires = null <br>2. + Key: enc-csf-key <br> class = oracle.security.jps.internal.credstore.PasswordCredentialImpl <br> desc = encryption key alias/password <br> name = orakey <br> pass = welcome1 <br> expires = null <br>3. + Key: keystore-csf-key <br> class = oracle.security.jps.internal.credstore.PasswordCredentialImpl <br> desc = keystore access password <br> name = n/a <br> pass = welcome1 <br> expires = null <br>4. + Key: test-appid-key <br> class = oracle.security.jps.internal.credstore.PasswordCredentialImpl <br> desc = null <br> name = weblogic <br> pass = welcome1 <br> expires = null</font></p> <p><font color="#800080" face="Courier New">### Map: IDCCS <br>1. + Key: ldap:1340771431089 <br> class = oracle.security.jps.internal.credstore.PasswordCredentialImpl <br> desc = null <br> name = ldap:1340771431089 <br> pass = MjcxN0M2ODREOEQ0RjZERg== <br> expires = null <br>2. + Key: db:1340771431083 <br> class = oracle.security.jps.internal.credstore.PasswordCredentialImpl <br> desc = null <br> name = db:1340771431083 <br> pass = MTlDQTE1N0EzQzE3REY1OA== <br> expires = null <br>3. + Key: hash:1340771431089 <br> class = oracle.security.jps.internal.credstore.PasswordCredentialImpl <br> desc = null <br> name = hash:1340771431089 <br> pass = MTk1MTk1QkQ4OUE2QzJBNw== <br> expires = null <br>4. + Key: proxy:1340771431089 <br> class = oracle.security.jps.internal.credstore.PasswordCredentialImpl <br> desc = null <br> name = proxy:1340771431089 <br> pass = M0IwMUZBQjNGQTUxNzk0OA== <br> expires = null</font> <br></p> <p>In a WebLogic domain, you will find that there is also a <font face="Courier New">cwallet.sso</font> found in <font size="2" face="courier new">$DOMAIN_HOME/config/fmwconfig/bootstrap</font>. </p> <p><font color="#800080" face="Courier New">Dumping store contents ...</font></p> <p><font color="#800080" face="Courier New">### Map: fks <br>1. + Key: master.key.0 <br> class = oracle.security.jps.internal.credstore.GenericCredentialImpl <br> desc = null <br> type = javax.crypto.spec.SecretKeySpec <br> algorith = AES <br> format = RAW <br> key material as hex = CB 45 4F B0 F8 26 FF 04 31 9F 48 DD 43 42 69 C7 <br> expires = null <br>2. + Key: current.key <br> class = oracle.security.jps.internal.credstore.GenericCredentialImpl <br> desc = null <br> type = java.lang.String <br> cred = master.key.0 <br> expires = null</font></p> <p><font color="#800080" face="Courier New">### Map: IntegrityChecker <br>1. + Key: kss <br> class = oracle.security.jps.internal.credstore.GenericCredentialImpl <br> desc = null <br> type = Array of byte <br> byte array as hex = AB 18 CD 76 6C 39 FE 46 A0 6D 1C F0 BC 8D 97 3A D1 64 BC 80 2C 33 64 8E AE C9 B1 63 88 BE 23 7C 37 2F 63 9D 55 2B 5E 8F 1E 08 0A 73 F1 A8 15 83 8F 24 3D 19 B8 79 6E 75 B2 1C 7F DB 72 FC AE BA 72 A3 62 62 27 29 EE DE <br> expires = null</font></p> <p>If you ever decide to reassociate the credential store with LDAP (e.g. using the <font size="2" face="Courier New">reassociateSecurityStore</font> command), you will find that the <font face="Courier New">bootstrap</font> <font face="Courier New">cwallet.sso</font> will contain credentials to access the LDAP store. Here is what my domain wallet files look like when using an LDAP credential store :-</p> <p><font size="2" face="Courier New">$DOMAIN_HOME/config/fmwconfig/cwallet.sso :</font></p> <p><font color="#800080" face="Courier New">Dumping store contents ...</font></p> <p><font color="#800080" face="Courier New">### Map: dip <br>1. + Key: cn=odisrv <br> class = oracle.security.jps.internal.credstore.PasswordCredentialImpl <br> desc = DIP Password <br> name = cn=odisrv,cn=Registered Instances,cn=Directory Integration Platform,cn=Products,cn=OracleContext <br> pass = YNDMSU1wP1WergcX <br> expires = null</font></p> <p><font color="#800080" face="Courier New">### Map: ODSMMap <br>1. + Key: ODSMKey.Wallet <br> class = oracle.security.jps.internal.credstore.PasswordCredentialImpl <br> desc = ODSM Key store password <br> name = ODSM <br> pass = 0000000000 <br> expires = null <br></font></p> <p><font size="2" face="Courier New">$DOMAIN_HOME/config/fmwconfig/bootstrap/cwallet.sso :</font></p> <p><font color="#800080" face="courier new">Dumping store contents ...</font></p> <p><font color="#800080" face="courier new">### Map: BOOTSTRAP_JPS <br>1. + Key: bootstrap_q6ShJcm89vO8N2oVoSFqTLuW6Sg= <br> class = oracle.security.jps.internal.credstore.PasswordCredentialImpl <br> desc = bootstrap user name and password <br> name = cn=orcladmin <br> pass = welcome1 <br> expires = null</font></p> <p><font color="#800080" face="courier new">### Map: fks <br>1. + Key: master.key.0 <br> class = oracle.security.jps.internal.credstore.GenericCredentialImpl <br> desc = null <br> type = javax.crypto.spec.SecretKeySpec <br> algorith = AES <br> format = RAW <br> key material as hex = 1C B5 89 2A 45 F2 BA A0 E5 C1 A8 F6 DE 6E FC 5A <br> expires = null <br>2. + Key: current.key <br> class = oracle.security.jps.internal.credstore.GenericCredentialImpl <br> desc = null <br> type = java.lang.String <br> cred = master.key.0 <br> expires = null</font></p> <p><font color="#800080" face="courier new">### Map: IntegrityChecker <br>1. + Key: kss <br> class = oracle.security.jps.internal.credstore.GenericCredentialImpl <br> desc = null <br> type = Array of byte <br> byte array as hex = FD 01 1E 54 D5 84 3B 9D AF DA 62 62 22 BB 7E A9 0C DB 08 A3 D9 71 9F A6 03 96 7F DF 29 69 37 55 60 5D 0E 32 EE 3A D0 D6 F2 A9 FD 58 DB 82 87 A0 98 D2 78 6A 47 48 E9 6B 86 3E 68 77 BC 17 01 B6 A0 BD 29 A2 3B E7 B7 73 <br> expires = null</font></p> <p> </p> <p>Have fun!</p> Matt (Brisbane/Australia)http://www.blogger.com/profile/16649081553551301779noreply@blogger.com2tag:blogger.com,1999:blog-2959498105239183642.post-91813345019415332702012-06-27T12:54:00.001+10:002012-06-27T12:54:32.594+10:00JDeveloper - RIDC JAX/WS SAML Connections to UCM / Content Server<p>I posted a few years back some really useful information on configuring the infrastructure of the JDeveloper 11g embedded WebLogic server to allow consumption of a JAX/WS Web Service protected by a WSM policy:</p> <p><a href="http://todayguesswhat.blogspot.com.au/2010/09/jdeveloper-11gr1-integrated-weblogic.html">http://todayguesswhat.blogspot.com.au/2010/09/jdeveloper-11gr1-integrated-weblogic.html</a></p> <p>Today I will extend this information with some useful tips for connecting an application that is deployed to the JDeveloper embedded WebLogic server to connect to a back-end Oracle Content Server (aka Oracle UCM; aka Oracle WebCenter Content Server) using JAX/WS protocol.</p> <p>Firstly, some background on the configuration of the UCM Server (11.1.1.6.0) and its associated domain:</p> <p>The UCM Server is deployed to a WebLogic Domain (<em>which I now will refer to as the UCM Domain</em>) configured with WSM policy manager; Such that, the oracle.wsmpm_template_11.1.1.jar template has been applied and subsequently configured to connect to the MDS schema installed by the RCU.</p> <p>The UCM Domain has been configured with an OID Authentication Provider.</p> <p>A keystore has been created for the UCM Domain and populated with key-certificate pair signed by the demo certificate authority (CA) that ships with WebLogic.  Note – don’t use this CA in production!</p> <p>oracle.wsm.security credential store entries have been created for the UCM Domain with the credentials / aliases required to access the keystore contents from above.</p> <p>GPA Policy has also been configured on the UCM Domain for “ws-service” using message protection service policy <font face="Courier New">oracle/wss11_saml_or_username_token_with_message_protection_service_policy</font> which supports username/password connections and saml connections.</p> <p>Below are some sample script calls that you can leverage to achieve the above UCM server configuration (assuming Linux):</p> <p><font color="#000080" size="2" face="Courier New">cd $DOMAIN_HOME/config/fmwconfig</font></p> <p><font color="#000080" size="2" face="Courier New"># Create service key-certificate pair signed by the demo CA cert "CertGenCA" with key password welcome1 <br />java utils.CertGen -certfile MyPublicCert -keyfile MyPrivateKey -keyfilepass welcome1 -cn "`hostname -f`"</font></p> <p><font color="#000080" size="2" face="Courier New"># Create and populate new service keystore default-keystore.jks with service key-certificate pair from above under alias orakey with keystore password welcome1, and alias password welcome1 <br />java utils.ImportPrivateKey -keystore default-keystore.jks -storepass welcome1 -certfile MyPublicCert.der -keyfile MyPrivateKey.der -keyfilepass welcome1 -alias orakey -keypass welcome1</font></p> <p><font color="#000080" size="2" face="Courier New"># Now add the root CA to the service keystore under the alias name CA <br />keytool -importcert -noprompt -alias CA -file $WL_HOME/server/lib/CertGenCA.der -keystore default-keystore.jks -storepass welcome1 <br /></font></p> <p><font color="#800080" size="2" face="Courier New">$MW_HOME/oracle_common/common/bin/wlst.sh</font></p> <p><font color="#800000" size="2" face="Courier New">wls_username="weblogic" <br />wls_password="welcome1" <br />wls_url="t3://localhost:7001" <br />connect(wls_username, wls_password, wls_url)</font></p> <p><font color="#800000" size="2" face="Courier New">createCred(map="oracle.wsm.security", key="keystore-csf-key", user="n/a", password="welcome1", desc="keystore access password") <br />createCred(map="oracle.wsm.security", key="sign-csf-key", user="orakey", password="welcome1", desc="signing key alias/password") <br />createCred(map="oracle.wsm.security", key="enc-csf-key", user="orakey", password="welcome1", desc="encryption key alias/password")</font></p> <p><font color="#800000" size="2" face="Courier New">serverConfig()</font></p> <p><font color="#800000" size="2" face="Courier New">domain_name=cmo.getName() <br />policy_set_name=domain_name+"-ws-service" <br />policy_type="ws-service" <br />resource='Domain("' + domain_name + '")' <br />policy="oracle/wss11_saml_or_username_token_with_message_protection_service_policy"</font></p> <p><font color="#800000" size="2" face="Courier New">beginRepositorySession() <br />createPolicySet(policy_set_name, policy_type, resource) <br />attachPolicySetPolicy(policy) <br />validatePolicySet() <br />commitRepositorySession() <br />disconnect() <br />exit() <br /></font></p> <p>Note above, we have leveraged a Message Protection service policy.  If we were to switch to a non-Message Protection service policy such as <font face="Courier New">oracle/wss_saml_or_username_token_service_policy</font> life would be much simpler.  However you need to be extremely careful with deploying such a policy in your enterprise.  You need to understand the consequences, as you may be creating yourself an easily exploitable security hole!!!</p> <p>The policy <font face="Courier New">oracle/wss_saml_or_username_token_service_policy </font>does NOT do signing/encryption, certificates are NOT used!. <br />Thus the client does not need a keystore configured.  Also, setting up trusted client DN for the issuer will have no affect. <br />This profile is in no way shape or form secure and should only be used on a closed internal network where no rogue client can exist. <br />A rogue client can simply leverage the client policy <font size="2" face="Courier New">oracle/wss10_saml_token_client_policy</font> and set <font size="2" face="Courier New">BindingProvider.USERNAME_PROPERTY</font> to any user they like and make a connection to the service.  </p> <p>I exploit this backdoor all the time here at Oracle with our fusion test / WAR room environments.  Effectively, you get an EVP-level visibility bug filed against you (which 98% of time is not a bug – at least in my code ;) ) – and you are forced to troubleshoot with a read-only account (if you are lucky) or some minimal obscure log messages.  Using this trick, I just check the UCM login service to see what policy is applied, and if no message protection policy is defined, I go get myself an admin connection and find out what going on before 92ing the bug!</p> <p>Anyway, back to the topic at hand.  We are using a Message Protection policy, so the client must have a valid keystore and key-certificate pair in use, and the client’s public certificate must be present in the server’s keystore.  The simplest mechanism in a non-security hardened test environment to achieve this requirement is to simply copy/clone the server’s keystore file to the client, and create identical credential store entries for the oracle.wsm.security properties.  In security hardened environments though, each client domain would have a unique signing/encryption keypair, and the client's associated public certificate must be manually imported in to the server domain's keystore.</p> <p>Let’s proceed with the JDeveloper embedded WebLogic client steps based on simple cloning of the server’s keystore! If you have not already created the default domain in JDeveloper, do so now, and fire up the server!  I’m running Oracle JDeveloper 11g Release 2 (11.1.2.1.0) on Windows XP.</p> <p>1. From the JDeveloper View menu, choose Application Server Navigator <br />2. Right click on Application Servers / IntegratedWebLogicServer, select "Create Default Domain..." <br />2. Right click on Application Servers / IntegratedWebLogicServer, select "Start Server Instance" <br /></p> <p>Step 1) Copy from the UCM server its default-keystore.jks file.</p> <p><font face="Courier New">/u01/app/oracle/product/Middleware/user_projects/domains/base_domain/config/fmwconfig/default-keystore.jks</font></p> <p>Copy this file to the <font face="Courier New">%DOMAIN_HOME%\config\fmwconfig</font> directory of your jdeveloper</p> <p>e.g. from Command Prompt:</p> <p><font color="#000080" size="2" face="Courier">SET DOMAIN_HOME=C:\JDeveloper\system11.1.2.1.38.60.81\DefaultDomain <br />%DOMAIN_HOME%\bin\setDomainEnv.cmd <br />cd %DOMAIN_HOME%\config\fmwconfig <br />## ftp binary get from UCM server the following file <br />/u01/app/oracle/product/Middleware/user_projects/domains/base_domain/config/fmwconfig/default-keystore.jks</font></p> <p>Step 2) Create credential store entries to access keys/certificates from keystore file above (make sure your domain is running!)</p> <p><font color="#800000" size="2" face="Courier New"><font color="#800080">%MW_HOME%\oracle_common\common\bin\wlst <br /></font>connect('weblogic','welcome1','t3://127.0.0.1:7101') <br />createCred(map="oracle.wsm.security", key="keystore-csf-key", user="n/a", password="welcome1") <br />createCred(map="oracle.wsm.security", key="sign-csf-key", user="orakey", password="welcome1") <br />createCred(map="oracle.wsm.security", key="enc-csf-key", user="orakey", password="welcome1") <br />exit()</font></p> <p>Step 3) Create GPA ws-client policy</p> <p><font color="#800000" size="2" face="Courier New"><font color="#800080">%MW_HOME%\oracle_common\common\bin\wlst <br /></font>connect('weblogic','welcome1','t3://127.0.0.1:7101')</font></p> <font color="#800000" size="2" face="Courier New"> <p><font color="#800000" size="2" face="Courier New">serverConfig()</font></p> <p>domain_name=cmo.getName() <br />policy_set_name=domain_name+"-ws-client" <br />policy_type="ws-client" <br />resource='Domain("' + domain_name + '")' <br />policy="oracle/wss11_saml_token_with_message_protection_client_policy"</p> </font> <p><font color="#800000" size="2" face="Courier New">beginRepositorySession() <br />createPolicySet(policy_set_name, policy_type, resource) <br />attachPolicySetPolicy(policy) <br />validatePolicySet() <br />commitRepositorySession() <br />disconnect() <br />exit() <br /></font></p> <p>Step 4) Configure JDeveloper Embedded WebLogic Domain to use OID Authentication Provider and OID server leveraged by the UCM server.</p> <p><font color="#800000"><font size="2"><font face="Courier New"><font color="#800080">%MW_HOME%\oracle_common\common\bin\wlst <br /></font>connect('weblogic','welcome1','t3://127.0.0.1:7101')</font></font></font></p> <p><font color="#800000" size="2" face="Courier New">edit() <br />startEdit() <br />securityRealm = cmo.getSecurityConfiguration().getDefaultRealm(); <br />print("Realm: " + securityRealm.getName())</font></p> <p><font color="#800000" size="2" face="Courier New">oid_provider_name="OIDAuthenticator" <br />oid_host="xxx.us.oracle.com" <br />oid_port=3060 <br />oid_user="cn=orcladmin" <br />oid_user_pwd="welcome1" <br />oid_userbase="cn=Users,dc=us,dc=oracle,dc=com" <br />oid_groupbase="cn=Groups,dc=us,dc=oracle,dc=com"</font></p> <p><font color="#800000" size="2" face="Courier New">oidAP = securityRealm.createAuthenticationProvider(oid_provider_name,"weblogic.security.providers.authentication.OracleInternetDirectoryAuthenticator") <br />oidAP.setHost(oid_host) <br />oidAP.setPort(oid_port) <br />oidAP.setPrincipal(oid_user) <br />oidAP.setCredential(oid_user_pwd) <br />oidAP.setUserBaseDN(oid_userbase) <br />oidAP.setAllUsersFilter("(&(uid=*)(objectclass=person))") <br />oidAP.setUserNameAttribute("uid") <br />oidAP.setUseRetrievedUserNameAsPrincipal(true) <br />oidAP.setGroupBaseDN(oid_groupbase) <br />oidAP.setControlFlag("SUFFICIENT")</font></p> <p><font color="#800000" size="2" face="Courier New">print("Provider order (existing) : ") <br />print securityRealm.getAuthenticationProviders() <br />daAP = securityRealm.lookupAuthenticationProvider("DefaultAuthenticator") <br />diaAP = securityRealm.lookupAuthenticationProvider("DefaultIdentityAsserter") <br />securityRealm.setAuthenticationProviders([oidAP,daAP,diaAP]) <br />print("Provider order (updated) : ") <br />print securityRealm.getAuthenticationProviders()</font></p> <p><font color="#800000" size="2" face="Courier New">print("Default Authenticator Control Flag (existing) : " + daAP.getControlFlag()) <br />daAP.setControlFlag("SUFFICIENT") <br />print("Default Authenticator Control Flag (updated) : " + daAP.getControlFlag())</font></p> <p><font color="#800000" size="2" face="Courier New">save() <br />activate(block="true") <br />disconnect() <br />exit()</font></p> <p>Step 5) Optional - update WLS admins ...</p> <p><font color="#800000"><font size="2"><font face="Courier New"><font color="#800080">%MW_HOME%\oracle_common\common\bin\wlst <br /></font>connect('weblogic','welcome1','t3://127.0.0.1:7101')</font></font></font></p> <p><font color="#800000" size="2" face="Courier New">securityRealm = cmo.getSecurityConfiguration().getDefaultRealm() <br />roleMapper = securityRealm.lookupRoleMapper("XACMLRoleMapper") <br />existing = roleMapper.getRoleExpression(None,"Admin") <br />print("WLS Admins (existing) : " +  existing)</font></p> <p><font color="#800000" size="2" face="Courier New">ecm_domain_admin_oid_group_cn="ECM Domain Administrators"</font></p> <p><font color="#800000" size="2" face="Courier New">roleMapper.setRoleExpression(None,"Admin",existing + "|Grp(" + ecm_domain_admin_oid_group_cn + ")") <br />print("WLS Admins (updated) : " +  roleMapper.getRoleExpression(None,"Admin"))</font></p> <p><font color="#800000" size="2" face="Courier New">disconnect() <br />exit()</font></p> <p>This should take care of all the infrastructure steps required.  Now it is just a matter of creating a little web application in JDeveloper that leverages RIDC to connect to the UCM Server using JAX/WS protocol and GPA client policy.</p> <p>File > New > General > Applications > Custom Application <br />Application Name: <font face="Courier New">RIDCJAXWSTest <br /></font>Project Name: <font face="Courier New">web</font> <br />Product Features: JSP and Servlets</p> <p>Right click on web project > New > Web Tier > Servlets > HTTP Servlet <br />Web Application Version: Servlet 2.5\JSP 2.1 <br />Servlet class: <font face="Courier New">RIDCTestServlet <br /></font>Servlet package: <font face="Courier New">web</font> <br />Mapping details: <br />Name: <font face="Courier New">RIDCTestServlet <br /></font>URL Pattern <font face="Courier New">/app</font></p> <p>Right click on web project > Project Properties > Libraries and Classpath <br />Add and select a new user library with classpath referencing <font face="Courier New">oracle.ucm.ridc-11.1.1.jar</font></p> <p><u>Servlet and XML Code</u></p> <p>web/RIDCTestServlet.java :-</p> <p><font size="2" face="Courier New">package web;</font></p> <p><font size="2" face="Courier New">import java.io.IOException; <br />import java.io.PrintWriter;</font></p> <p><font size="2" face="Courier New">import java.util.Map;</font></p> <p><font size="2" face="Courier New">import javax.servlet.*; <br />import javax.servlet.http.*;</font></p> <p><font size="2" face="Courier New">import oracle.stellent.ridc.IdcClient; <br />import oracle.stellent.ridc.IdcClientException; <br />import oracle.stellent.ridc.IdcClientManager; <br />import oracle.stellent.ridc.IdcContext;</font></p> <p><font size="2" face="Courier New">import oracle.stellent.ridc.model.DataBinder; <br />import oracle.stellent.ridc.model.DataObject;</font></p> <p><font size="2" face="Courier New">import oracle.stellent.ridc.protocol.ServiceResponse;</font></p> <p><font size="2" face="Courier New">public class RIDCTestServlet extends HttpServlet <br />{ <br />  private static final String url = "</font><a href="http://ucmserver.com:16200/idcnativews"><font size="2" face="Courier New">http://ucmserver.com:16200/idcnativews</font></a><font size="2" face="Courier New">"; <br />    <br />  private IdcClient m_idcClient = null;</font></p> <p><font size="2" face="Courier New">  public void init(ServletConfig config) throws ServletException <br />  { <br />    super.init(config); <br />    IdcClientManager clientManager = new IdcClientManager(); <br />    try <br />    { <br />      m_idcClient = clientManager.createClient(url); <br />    } <br />    catch (IdcClientException e) <br />    { <br />      e.printStackTrace(); <br />      throw new RuntimeException(e); <br />    }</font></p> <p><font size="2" face="Courier New">    // enable verbose ridc logging to console <br />    oracle.stellent.ridc.common.log.LogFactory.setLogProvider( <br />      new oracle.stellent.ridc.common.log.simple.SimpleLogProvider()); <br />  }</font></p> <p><font size="2" face="Courier New">  protected void doGet(HttpServletRequest request, HttpServletResponse response) <br />    throws ServletException, IOException <br />  { <br />    doPost(request, response); <br />  }</font></p> <p><font size="2" face="Courier New">  protected void doPost(HttpServletRequest request, HttpServletResponse response) <br />    throws ServletException, IOException <br />  { <br />    PrintWriter out = response.getWriter(); <br />    out.print("<html><head><title>RIDC Test Servlet</title></head><body><pre>"); <br />    <br />    try <br />    { <br />    <br />      out.print("RIDC Version: " + m_idcClient.getVersion() + "<br/><hr/>"); <br />      <br />      String username = request.getRemoteUser(); <br />      out.print("request.getRemoteUser() = " + username + "<br/><hr/>"); <br />      <br />      // we are using SAML <br />      IdcContext userContext = userContext = new IdcContext(username);</font></p> <p><font size="2" face="Courier New">      // get the binder <br />      DataBinder binder = m_idcClient.createBinder(); <br />  <br />      // populate the binder with the parameters <br />      binder.putLocal("IdcService", "PING_SERVER");</font></p> <p><font size="2" face="Courier New">      out.print("Sending PING_SERVER request ...<br/>");</font></p> <p><font size="2" face="Courier New">      // execute the request <br />      ServiceResponse resp = m_idcClient.sendRequest(userContext, binder); <br />  <br />      // get the binder - get a binder closes the response automatically <br />      DataBinder responseBinder = resp.getResponseAsBinder(); <br />      <br />      out.print("<hr/>PING_SERVER response ...<br/>");</font></p> <p><font size="2" face="Courier New">      DataObject localData = responseBinder.getLocalData(); <br />      for (Map.Entry<String, String> entry : localData.entrySet()) <br />      { <br />        out.print(entry.getKey() + "=" + entry.getValue() + "<br/>"); <br />      } <br />      <br />      m_idcClient.logout(userContext); // only new versions of RIDC support this <br />    } <br />    catch (Exception e) <br />    { <br />      out.print("Exception: <br/><hr/>"); <br />      e.printStackTrace(out); <br />    } <br />    finally <br />    { <br />      out.print("</pre></body></html>"); <br />    } <br />  } <br />}</font></p> <p>WEB-INF/web.xml :-</p> <p><font size="2" face="Courier New"><?xml version = '1.0' encoding = 'UTF-8'?> <br /><web-app xmlns="</font><a href="http://java.sun.com/xml/ns/javaee"><font size="2" face="Courier New">http://java.sun.com/xml/ns/javaee</font></a><font size="2" face="Courier New">" <br />         xmlns:xsi="</font><a href="http://www.w3.org/2001/XMLSchema-instance"><font size="2" face="Courier New">http://www.w3.org/2001/XMLSchema-instance</font></a><font size="2" face="Courier New">" <br />         xsi:schemaLocation="</font><a href="http://java.sun.com/xml/ns/javaee"><font size="2" face="Courier New">http://java.sun.com/xml/ns/javaee</font></a><font size="2" face="Courier New"> </font><a href="http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"><font size="2" face="Courier New">http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd</font></a><font size="2" face="Courier New">" <br />         version="2.5"> <br />  <servlet> <br />    <servlet-name>RIDCTestServlet</servlet-name> <br />    <servlet-class>web.RIDCTestServlet</servlet-class> <br />  </servlet> <br />  <servlet-mapping> <br />    <servlet-name>RIDCTestServlet</servlet-name> <br />    <url-pattern>/app</url-pattern> <br />  </servlet-mapping></font></p> <p><font size="2" face="Courier New">  <security-constraint> <br />    <web-resource-collection> <br />      <web-resource-name>Constraint-0</web-resource-name> <br />      <url-pattern>/*</url-pattern> <br />    </web-resource-collection> <br />    <auth-constraint> <br />      <role-name>valid-users</role-name> <br />    </auth-constraint> <br />  </security-constraint></font></p> <p><font size="2" face="Courier New">  <login-config> <br />    <auth-method>BASIC</auth-method> <br />    <realm-name>myrealm</realm-name> <br />  </login-config></font></p> <p><font size="2" face="Courier New">  <security-role> <br />    <role-name>valid-users</role-name> <br />  </security-role> <br /></web-app></font></p> <p> </p> <p>WEB-INF/weblogic.xml :-</p> <p><font size="2" face="Courier New"><?xml version="1.0" encoding="UTF-8"?> <br /><wls:weblogic-web-app xmlns:wls="</font><a href="http://xmlns.oracle.com/weblogic/weblogic-web-app"><font size="2" face="Courier New">http://xmlns.oracle.com/weblogic/weblogic-web-app</font></a><font size="2" face="Courier New">" xmlns:xsi="</font><a href="http://www.w3.org/2001/XMLSchema-instance"><font size="2" face="Courier New">http://www.w3.org/2001/XMLSchema-instance</font></a><font size="2" face="Courier New">" xsi:schemaLocation="</font><a href="http://java.sun.com/xml/ns/javaee"><font size="2" face="Courier New">http://java.sun.com/xml/ns/javaee</font></a><font size="2" face="Courier New"> </font><a href="http://java.sun.com/xml/ns/javaee/web"><font size="2" face="Courier New">http://java.sun.com/xml/ns/javaee/web</font></a><font size="2" face="Courier New">-</font></p> <p><font size="2" face="Courier New">app_2_5.xsd </font><a href="http://xmlns.oracle.com/weblogic/weblogic-web-app"><font size="2" face="Courier New">http://xmlns.oracle.com/weblogic/weblogic-web-app</font></a><font size="2" face="Courier New"> </font><a href="http://xmlns.oracle.com/weblogic/weblogic-web-app/1.0/weblogic-web-app.xsd"><font size="2" face="Courier New">http://xmlns.oracle.com/weblogic/weblogic-web-app/1.0/weblogic-web-app.xsd</font></a><font size="2" face="Courier New">"></font></p> <p><font size="2" face="Courier New">  <wls:security-role-assignment> <br />    <wls:role-name>valid-users</wls:role-name> <br />    <wls:principal-name>users</wls:principal-name> <br />  </wls:security-role-assignment></font></p> <p><font size="2" face="Courier New">  <!-- <br />  the users group is automatically made available to the security realm; <br />  this group represents the set of all authenticated users. <br />  Any user that successfully authenticates itself is a member of the users group. <br />  --> <br /></wls:weblogic-web-app></font></p> <p> </p> <p>Sample Output:  (after authenticating as user ecmadmin)</p> <pre><font size="2">RIDC Version: 11.1.1.6.3.7848<br /><hr />request.getRemoteUser() = ecmadmin<br /><hr />Sending PING_SERVER request ...<br /><hr />PING_SERVER response ...<br />UserDateFormat=iso8601<br />UserTimeZone=UTC<br />ClientEncoding=UTF-8<br />IdcService=PING_SERVER<br />dUser=ecmadmin<br />refreshSubjects=<br />blDateFormat=yyyy-MM-dd HH:mm:ssZ!tUTC!mAM,PM<br />ECID-Context=1.8987b47a9955d4f6:328ee95f:1382badfe3d:-8000-0000000000000039;kXjE<br />refreshMonikers=<br />changedSubjects=<br />refreshSubMonikers=<br />blFieldTypes=xForceFolderSecurity text,xPartitionId text,dInDate date,xReadOnly text,xPartnerLevel bigtext,dMessage message,xInhibitUpdate text,xPartnerProgram bigtext,dCreateDate date,xWebFlag text,xHidden text,xPartnerType bigtext,dReleaseDate date,StatusMessage message,xCollectionID int,xStorageRule text,dOutDate date,xExternalDataSet bigtext,xComments memo,xIdcProfile text<br />NoHttpHeaders=0<br />changedMonikers=<br />idcToken=<br />StatusMessage=You are logged in as 'ecmadmin'.<br />hasUserAccessChanged=1<br />IsJava=1<br />localizedForResponse=1</font></pre> Matt (Brisbane/Australia)http://www.blogger.com/profile/16649081553551301779noreply@blogger.com2tag:blogger.com,1999:blog-2959498105239183642.post-19428255775971709762012-06-27T10:11:00.001+10:002012-06-27T10:11:41.328+10:00Spaces wrongly added in OCR text of PDF generated by Abbyy FineReader - Solution<p>In my search for the holy grail of document scanning and management software, I’ve been trialing Abbyy FineReader 11 and Adobe Acrobat X to see if either offer any benefits over Lucion FileCenter on the scanning front.  FileCenter is a nice piece of software for managing of scans, but it falls short in my opinion on the OCR front.  </p> <p>What I’m hoping to find in a product is something that offers a high level of OCR accuracy, and the ability to easily correct mistakes with a quick and simple interface (preferably all controlled by keyboard commands for improved processing time).</p> <p>I’m after a product that could learn / be trained to detect document types and only OCR the data of actual interest.</p> <p>I’m after a product that easily and automatically recognize document paper size and allows simple and effective trim/deskew/cleanup etc as necessary.</p> <p>Anyway, these requirements culminated in my testing of FineReader and Acrobat X.</p> <p>Acrobat X has the potential to be a useful piece of software, but it surprisingly performed quite average on the OCR front with the documents that I threw at it.  The OCR results were average – and that is being kind.  What astonished me however, was the product’s inability to provide a simple mechanism to correct the OCR mistakes.  I had to google search for a way to achieve it; and if anything, the results returned were more a workaround/hack than out-of-the-box intended functionality.  What it did do nicely was the simple one click scan to PDF.  It was just a pity the OCR let it down.</p> <p>Next up was FineReader 11.  This software appeared to excel on the OCR front, and also allow a simple verification mechanism to correct issues detected.  It however was not without its warts.   Features that could turn this very good software in to excellent software:</p> <ol> <li>The ability to perform all document verification/correction steps completely using keyboard with absolutely no mouse interaction required. </li> <li>The ability to save scanning profiles for specific document types.  For example what type of fonts to expect, what paper size, what spelling mistakes/words to ignore.</li> <li>The ability for spell checker / verifier to ignore single letters that appear as part of a word grouping.  For example,  John Smith is fine.  But it sees an issue with the “J” in J Smith.  I should be able to add “J Smith” to my dictionary ignore list.</li> <li>The ability to completely skip OCR on specific pages of a document without silly workarounds.  Currently, you have to add a scan region on the page, and read the page.</li> <li>The ability to better recognize font families and the like.</li> </ol> <p>One major flaw I found in the software, fortunately appears to have a cure. I was hoping others would have run in to it and provided a solution in some type of forum, but alas that was not the case.  Abbyy – if you are reading this post by any chance, I would strongly suggest you consider making a public support forum for people to discuss your software and offer tips etc.  It should definitely lead to more sales if people can find solutions to problems they encounter with your software.</p> <p>Anyway, the <strong>issue </strong>I ran in to was the following:</p> <p><em>When converting scan to PDF, words had invalid spacing/padding added not present in either the OCR text in Abbyy, or for that matter, in other output options such as HTML etc.</em></p> <p><strong>The cause:</strong></p> <p><u>The Windows machine likely does not have the matching font installed, and Abbyy likely not configured to leverage the font.</u></p> <p><strong>The solution:</strong></p> <p>First attempt to identify the font leveraged by the document.  If only a scan is available, extract some words containing sufficient sample characters and head on over to <a href="http://www.myfonts.com/WhatTheFont/">http://www.myfonts.com/WhatTheFont/</a> and attempt to determine the font.</p> <p>If you can get hold of an electronic PDF of the same file (or similar from same provider), you should be able to go to Acrobat > File > Properties dialog > Fonts tab.  If an electronic version of the document was available, within Acrobat you will likely see that the font has been embedded, but only <br />a portion of the font was embedded (such that the characters leveraged by the document, and nothing else).</p> <p>Using tools such as FontForge and mupdf-1.0-tools-windows.zip, you can potentially extract the embedded fonts subsets, and try and convert these to TTF.  But it is likely the TTF won't be sufficient.  Your best bet is to try and find the original TTF on the net.  You may have to purchase it.</p> <p>One other issue, is that even if you can find the TTF, chances are licensing flags on the TTF will prevent it from being embedded in a PDF document. <br />To circumvent this, you can leverage ttfpatch and set the fsType flag to 0, meaning: Installable embedding allowed, fonts may be embedded in documents and permanently installed on the remote system.</p> <p>Install the font then in the <windows>\fonts directory, and open Abbyy. <br />From Tools > Options > Read tab > Fonts > ensure the font is selected (and thus available for OCR).</p> <p>If saving the file as PDF/A (which embeds fonts), and an error message appears stating the font is restricted from being embeeded, first ensure the fsType flag has been reset to 0, and also try and clear out Abbyy font cache.  Delete directory <br />C:\Documents and Settings\All Users\Application Data\ABBYY\FineReader\11.00\FontCache</p> <p>Also, it is worthwhile installing Microsoft Typography Font Properties Extension <br /><a href="http://www.microsoft.com/typography/FreeToolsOverview.mspx">http://www.microsoft.com/typography/FreeToolsOverview.mspx</a> <br />This will allow you to right click > properties on a TTF file, and see the Embedding restrictions on the file. <br /> <br />Good luck! </p> Matt (Brisbane/Australia)http://www.blogger.com/profile/16649081553551301779noreply@blogger.com3tag:blogger.com,1999:blog-2959498105239183642.post-54007251253937468652012-02-17T14:38:00.001+10:002012-02-17T14:38:13.557+10:00Disable Acrobat X Welcome Screen<p>Create and invoke the following registry file to disable the annoying welcome screen in Acrobat X Pro on Windows:</p> <p><font face="Courier New">Disable_Acrobat_X_Pro_Welcome_Screen.reg </font> contents:</p> <p><font color="#800080" size="2" face="Courier New">REGEDIT4</font></p> <p><font color="#800080" size="2" face="Courier New">[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Adobe\Adobe Acrobat\10.0\FeatureLockDown\cWelcomeScreen] <br />"bShowWelcomeScreen"=dword:00000000</font></p> Matt (Brisbane/Australia)http://www.blogger.com/profile/16649081553551301779noreply@blogger.com0tag:blogger.com,1999:blog-2959498105239183642.post-36072211055933730802011-04-19T10:45:00.001+10:002011-04-19T13:29:10.985+10:00dircolors – color ls unix directory listings – danger of overriding shell built-ins<p>I ran in to a crazy issue today that stemmed from a built-in function override configuration setting that I have leveraged for the best part of a decade. I’m amazed that this is the first time I actually noticed it!</p> <p>I have a environment setup script for one of my installations like the following ... <br /> <br /><font face="courier new" size="2">export MW_HOME=/u01/app/oracle/product/Middleware <br />export JAVA_HOME="`ls -d ${MW_HOME}/jdk16*`" <br />export PATH=$JAVA_HOME/bin:$PATH <br /></font> <br />The JDK path is not hardcoded as this is a pre-release environment, and the JDK is often patched and the directory name changes as a result. </p> <p>A script I was invoking had a simple test like the following:</p> <p><font face="Courier New" color="#800080" size="2">#!/bin/sh <br />if [ x${PATH} != x ]; then <br />    PATH=$ORACLE_HOME/bin:$PATH <br />fi <br /></font> <br />When I ran the shell script from plain vt100 terminal, everything worked perfectly.  When I fired the script up from an xterm running inside VNC, I hit an error like the following:</p> <p><font face="Courier New" color="#ff0000">./test.sh: line 2: [: too many arguments <br /></font> <br />The reason for the error, the string JAVA_HOME ended up containing ANSI characters when run from xterm. </p> <p>For example, from an XTERM <br /> <br /><font face="Courier New" color="#0000ff" size="2">echo `ls -d /tmp` > file.txt <br /></font> <br /><font face="Courier New" size="2">file.txt</font> contained ... <font face="Courier New">^[[00m^[[00;34m/tmp^[[00m ^[[m <br /></font> <br />, whereas from the VT100 terminal: <br /> <br /><font face="Courier New" color="#0000ff" size="2">echo `ls -d /tmp` > file.txt </font> <br /> <br /><font face="Courier New" size="2">file.txt</font> contained ... <font face="Courier New">/tmp <br /></font> <br />It turns out, I had a zsh function setup-for/overriding the <font face="Courier New">ls</font> command so as to use color listings when invoked from an xterm by passing the <font face="Courier New">--color</font> option.  The function was as follows:</p> <p><font face="Courier New" color="#800080" size="2">ls () <br />{ <br />  [[ $TERM = (<b class="moz-txt-star"><span class="moz-txt-tag">*</span>xterm<span class="moz-txt-tag">*</span></b>|ansi|linux|dtterm) ]] && set -- <strong>--color</strong> "$@" <br />  command ls -v "$@" <br />} <br /></font> <br />The fix was to change the <font face="courier new">--color</font> option to be <font face="courier new">--color=auto</font> so that the ANSI escape sequences were only output if the standard output is a terminal (but not for things like scripts etc).</p> <p><font face="Courier New" color="#800080" size="2">ls () <br />{ <br />  [[ $TERM = (<b class="moz-txt-star"><span class="moz-txt-tag">*</span>xterm<span class="moz-txt-tag">*</span></b>|ansi|linux|dtterm) ]] && set -- <strong>--color=auto</strong> "$@" <br />  command ls -v "$@" <br />} <br /></font> <br />Moral of the story, be extremely careful using shell functions to override standard built-in commands, and make sure you understand the impact of switches! <br /></p> <p><strong><u>Update</u></strong></p> <p>So my good mate and colleague Dragos states:</p> <p>You should not rely on '<font face="Courier New" color="#800080">ls</font>' (or any other command), as it can be an alias or function to anything else.</p> <p>The fix is in your environment script to use <font face="Courier New" color="#800080">/bin/ls</font> or <font face="Courier New" color="#800080">=ls</font>, yes "equal sign ls"</p> <p>There's a way to force filename expansion too:</p> <p><font face="Courier New">JAVA_HOME=(${MW_HOME}/jdk16*) </font></p> Matt (Brisbane/Australia)http://www.blogger.com/profile/16649081553551301779noreply@blogger.com0tag:blogger.com,1999:blog-2959498105239183642.post-11711945492661762442011-04-07T13:27:00.001+10:002011-04-07T13:32:16.307+10:00Online Open Port Check Tool<p>The RCCB / Residual Current Device in my place has a nasty habit of tripping whenever the house is vacant for more than a couple of days. For the remaining 99% of the time that I’m home, it never trips.  I haven’t been able to isolate exactly what triggers it, but upon my return I’m faced with a fridge/freezer full of ruined food.</p> <p>Anyway, given the electricity is completely lost when this RCCB triggers, I really need a simple tool that polls my router (like a ping) to see if it responds, and if not, gives me some type of warning.</p> <p>I run OpenVPN on the router over TCP port 443, and it appears the various online port scanning tools are able to detect the presence of the port being open.</p> <p>The following is a list of sites that I’ve found that <strong>allow</strong> one to scan a specific <strong>remote</strong> host and port (that differs to the host/proxy of the incoming client):</p> <p><a href="http://ping.eu/port-chk/">http://ping.eu/port-chk/</a> <br /><a href="http://www.subnetonline.com/pages/network-tools/online-port-scanner.php">http://www.subnetonline.com/pages/network-tools/online-port-scanner.php</a> <br /><a href="http://www.digitalcoding.com/tools/open-port-check.html">http://www.digitalcoding.com/tools/open-port-check.html</a> <br /><a href="http://www.yougetsignal.com/tools/open-ports/">http://www.yougetsignal.com/tools/open-ports/</a> <br /></p> <p>My favourite is the latter, as it allows you to potentially bookmark the location; for example, to check if google port 80 open:</p> <p><a href="http://www.yougetsignal.com/tools/open-ports/php/check-port.php?portNumber=80&remoteAddress=www.google.com">http://www.yougetsignal.com/tools/open-ports/php/check-port.php?portNumber=80&remoteAddress=www.google.com</a></p> <p>If anyone knows a site out there that offers remote port polling with SMS/email notification in the event the service is down, let me know!</p> <p>I may have to build something otherwise on Google App Engine using the URL fetch capabilities </p> <p>Here is the API<a href="http://www.yougetsignal.com/tools/open-ports/php/check-port.php?portNumber=443&remoteAddress=mshannon.blogsite.org"> </a>  <a href="http://code.google.com/appengine/docs/java/urlfetch/overview.html">http://code.google.com/appengine/docs/java/urlfetch/overview.html</a></p> Matt (Brisbane/Australia)http://www.blogger.com/profile/16649081553551301779noreply@blogger.com0