Local IMAP4 server on Windows (cygwin)

When moving from Thunderbird to e.g. MS Outlook 2007 there are issues how to migrate the emails from one format to another. There are two main option:

1. Use 3rd party tool to migrate the Thunderbird folders to PST format or another format that Outlook supports. There are some free utilities that convert the files to EML format.

2. Setup a local IMAP4 server that acts only locally to serve your folders/emails. When IMAP server is in your local workstation then use that as your "local Mail Account" to extend your email capabilities from the central server.

 

Looked at the option 1) and quickly realized that I could spend ages converting my emails more or less manually to Outlook format. Forgot that.

Moved to option 2) which is actually a very nice approach to virtualize my emails as a generic service that could be used anywhere, not just my email client (being that Thunderbird or Outlook).

 

How to do this was another issue. There were bits and pieces on the Internet but none of the documents explained exactly what I should do. Finally, after getting this working I decided to document these steps, at least to serve my own memory in the future.

Here are the steps for Windows users to setup open source IMAP4/POP3 server on your workstation. Hopefully I didn't forget anything.

 

A word of warning. IMAP4/POP3 server shipped with Cygwin (uw-imap/uw-ipop3) should not be used in production use. It seems to work but has some limitations, like not handling subfolders elegantly and some other rarerities. So, take it as it is, and adopt your way of working.

Yes, I know there are other alternatives but this is not a subject of this post.

 

1. Install cygwin, inettools and OpenSSL (as cygwin packages)

If cygwin isn't familiar, go to http://www.cygwin.com/.

 

2. After installing cygwin + tools open up the bash command line

Navigate to the directory you installed cygwin to and enter cygwin.bat. This should bring you Unix like shell and when you enter commands described below, you should be fine.

 

3. Make sure you have following lines in your /etc/inetd.conf

In other words, edit the file mentioned above.

pop3  stream  tcp  nowait  root   /usr/sbin/uw-ipop3d
imap  stream  tcp  nowait  root   /usr/sbin/uw-imapd
pop3s  stream  tcp  nowait  root   /usr/sbin/uw-ipop3d
imaps  stream  tcp  nowait  root   /usr/sbin/uw-imapd

 

4. Add following lines to /etc/services

imaps             993/tcp
pop3s             995/tcp

5. Create server certificates for the IMAP/POP3 SSL connection

UW-IMAP uses SSL certificates from Cygwin:s /usr/ssl/certs directory (or whichever directory is defined in /usr/ssl/openssl.cnf as “dir” variable) . Below are sample command to creat self-signed certificates for your own purpose or development/testing:

cd /usr/ssl/certs

openssl req -new -x509 -nodes -out uw-imapd.pem -keyout uw-imapd.pem -days 3650

openssl req -new -x509 -nodes -out uw-ipop3d.pem -keyout uw-ipop3d.pem -days 3650

Please note that you MUST use exactly those file names to get things working correctly.

 

NOTE: Please remember to enter your workstation name or "localhost" in the question for "Common name (eg. YOUR name) []:". If you entered your own name you will have following error message every time you login to see your emails in IMAP server:

 

6. Install inetd daemon as Windows service

/usr/sbin/inetd    --install-as-service

If above approach doesn’t work properly, try out following:

cygrunsrv -I inetd -d "CYGWIN inetd" -p /usr/sbin/inetd -a -d -e CYGWIN="tty ntsec"

7. Refresh Windows users to the local passwd and group files

mkpasswd  --local   >   /etc/passwd
mkgroup  --local   >   /etc/group

After this when you connect to IMAP server you can use your Windows account username and password to login.

 

8. If using exim and imap together, tie these directories together with symbolic links

cd /var/spool
ln -s /var/spool/mail /var/mail
chmod   1777   /var/mail

 

9. Start the IMAP server

net start inetd

Stopping is done like this: net stop inetd

 

10. Connecting to IMAP server using SSL connection

By default the binaries delivered with cygwin have IMAP server in the most standard mode, plaintext login disabled by default. This means you need to set up IMAPS connection to the IMAP server. The first time you connect, you will get following error message if using self-signed certificates:

 

After you choose "Accept the certificate permanently" and press OK, you should have login screen to enter your username and password. Next time you login, this nag screen doesn't show up since you accepted your own signed certificate permanently.

On Outlook, you will get a warning stating that the self signed root certificate for (e.g. localhost) doesn't exist. You fix this by importing your self signed root certificate to Internet Explorer (Tools -> Internet Options -> Content -> Certificates -> Trusted Root Certification Authorities Tab -> Import). Before this you must convert your self signed certificate to the format that IE understands:

/usr/ssl/certs

openssl x509 -in uw-imapd.pem -inform PEM -outform DER -out uw-imapd.crt

You need to import this uw-imapd.crt file on IE.

 

11. All is fine and the world is a better place to live

Troubleshooting

IMAP doesn’t work after updating cygwin stack.

Symptom of this might be that you just get empty or no response from IMAP server. Inetd might be accepting the request but no folders or message content is shown. Make sure that SSL setup is done correctly. Sometimes cygwin update will overwrite the openssl.cnf file and change the certificates directory location. Double check the setup and directories.

Oracle Applications Adapter: IREP File not present

I have been playing with the Oracle Applications adapter. When connecting to EBS Rel12 the first time you might get following dialog prompting to download the irep file as local copy. When you press "Yes", adapter wizard will download the irep repository content to your local file.

Getting IREP File not present repeatedly

Once you have downloaded the repository you should be fine. But you might be getting this same dialog file coming to you, time after time. The issue is not that you haven't received the local repository. I noticed that the reason is because of your database connection name is in uppercase and the wizard is looking for file name in lowercase.

The workaround to get over this repeated error message is to rename the local irep file to all lowercase, like this:

An example:

ren irep_data_EBS12DEV_20081103_2143.dat irep_data_ebs12dev_20081103_2143.dat

 

After this, you should have another wizard page in the Oracle Applications Adapter wizard page:

Adding encoding -processing Instruction to XML When Using File Adapter

By default when using Oracle File Adapter to produce XML documents, no encoding="<charset>" attribute is used in the <?xml version="1.0"?> processing instruction.

Default PI:

<?xml version="1.0" ?>

and wanted PI:

<?xml version="1.0" encoding="UTF-8" ?>

 

The way to make this work (at least in 10.1.3.4) is to update the XML schema to include two new options in the schema definition.

           xmlns:nxsd="http://xmlns.oracle.com/pcbpel/nxsd"
           nxsd:encoding="UTF-8">

So, adding these attributes will make the schema look like:

<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" targetNamespace="http://www.fi.oracle.com/harri.kaukovuo"
           elementFormDefault="qualified" attributeFormDefault="unqualified"
           xmlns:nxsd="http://xmlns.oracle.com/pcbpel/nxsd"
           nxsd:encoding="UTF-8">
<xs:element name="ORDERS">
...

</xs:element>

</xs:schema>

Great utility to free up disk space: JDiskReport

There is not enough of disk space. No matter how large hard disk you get, it will get filled up eventually.

 

After having only 20MB of free space on my C: -drive, I finally take a closer look at the utilities that can point me the directories and files that are reserving the most of the disk space. I found this great utility:

http://www.jgoodies.com/freeware/jdiskreport/index.html

Good looking, graphical, Java -based, free and easy to use utility to show exactly what are the directories and files consuming the most of your disk space.

After scanning thru my C: and D: drives I could purge almost 13 GB of meaningless files.

Voi räkä!

I am running Google Chrome Beta and faced an issue with the browser. The error message is a bit interesting when shown in Finnish language:

Funny indeed :)

JMSUtils

There is a built-in utility to browse/move/delete(drain) messages from JMS queues and topics inside OracleAS OC4J. Starting of the utility has changed a little bit from version 10.1.2 but the usage is the same.

For those that want the fast way of starting up this utility, here is a sample Windows command file:

Create a new file, e.g. called JMSUtils.cmd and the contents:

@REM JMSUtils.cmd
@REM    This script will launch OracleAS OC4J 10.1.3. JMS command line utility
@REM History:
@REM   4.6.2008 Harri Kaukovuo, Oracle Finland Oy
@set J2EE_HOME=D:\product\10.1.3.1\OracleAS_1\j2ee\home
@setlocal
@set CLASSPATH=%J2EE_HOME%\oc4j.jar;%J2EE_HOME%\oc4j-api.jar;%J2EE_HOME%\oc4jclient.jar;%J2EE_HOME%\rmic.jar;%J2EE_HOME%\lib\adminclient.jar;%J2EE_HOME%\lib\connector.jar;%J2EE_HOME%\lib\javax77.jar;%J2EE_HOME%\lib\jmxri.jar;%J2EE_HOME%\lib\oc4j-internal.jar
@java com.evermind.server.jms.JMSServerUtils -username oc4jadmin -password yourpassword -port 12601 %*
@endlocal

You need to replace the value for J2EE_HOME to point to your OracleAS OC4J home directory. By default this utility points to the local host, but you need to connect remote OC4J please add extra "-host myhost.com" as startup uption. To determine the right JMS server port, please see the following chapter.

Determing JMS server port

OPMN

Depending on the installation (SOA Suite basic, advanced, WebCenter, standalone OC4J), the JMS server setting might be different. One of the easiest way to determine this when running a real OracleAS installation is to use OPMN command line tool:

opmnctl status -fmt %por

Processes in Instance: soasuite.hkaukovu-fi
-----
ports
-----
N/A
jms:12601,http:8888,rmis:12701,rmi:12401

As you can see, the JMS server port among others is reported as an output. You should use the jms:XXXX value in the JMSUtils.cmd command script as -port option.

opmnctl is found under $ORACLE_HOME/opmn/bin.

 

Standalone OC4J

If you are running standalone OC4J (not installed using Oracle Universal Installer), you should look for $OC4J_HOME/j2ee/home/config/jms.xml file. The entry

<jms-server port="9129">

will show you the port number.

In the above sample the "home" in the path $OC4J_HOME/j2ee/home/config points to the default OC4J instance name "home". In case you have multiple OC4J instances you might be different subdirectories for each instance (like oc4j_soa etc).

Please be aware that if you use OPMN to control the instance, the value in jms.xml will be overriden by OPMN driven values derived from opmn.xml "<port id="jms" range="12601-12700"/>".

ODI Agent and AD4J Diagnostics Console

I was playing with Oracle's new java application diagnostics tool called Oracle Application Diagnostics for Java (AD4J).

http://www.oracle.com/technology/software/products/oem/htdocs/jade.html

http://www.oracle.com/technology/products/oem/pdf/wp_productionappdiagnostics.pdf

This is great tool to monitor the guts of Java or J2EE engine.

 

AD4J manuals describe how to install the JADE agent on OracleAS or BEA WebLogic, also decribing how to configure standard Java application with agent.

For those that want to monitor Oracle Data Integrator (ODI) agents, here are the steps in details.

1. Download the "Java Agent ZIP File"

2. Place the zip file on ODI installation in $ODIHOME/drivers -directory.

3. Edit the agent.bat (or agent.sh on Unix) to replace the

%ODI_JAVA_START% oracle.odi.Agent %*

with

%ODI_JAVA_START% jamagent.jamrun jamconshost=localhost jamconsport=3600 jamjvmid=1000 oracle.odi.Agent %*

Here you should replace jamconshost parameter value with the hostname that has AD4J console running. Also if jamconsport is different than 3600, change the value. Jamjvmid is the identifier that shows up in the JVM list as "port" identifying the JVM from others:

4. Once the agent.bat or agent.sh is modified (alternatively you made a copy of the agent startup script), you can start it and should see the ODI agent JVM showing up in the AD4J console.

Adding Oracle ODI Agent execution under OPMN control

I was talking to a customer and explaining opmn and it's role acting as main process control for OracleAS and custom components. While talking also about Oracle Data Integrator (ODI) agents and how to start them up, it occured to my mind to test how to place agent startup/shutdown under opmn control.

Adding your own scripts and executables under opmn control as part of the whole OracleAS package is supported and described in the OPMN manual.

Here is how it goes when adding ODI Agent:

Modify and edit following XML fragment and paste it on $ORACLE_HOME/opmn/conf/opmn.xml

....

<ias-component id="OdiAgent" status="enabled" id-matching="false">
    <environment>
       <variable id="ODI_JAVA_HOME"
                 value="D:\product\10.1.3.1\OracleAS_1\jdk" append="false"/>
       <variable id="ODI_HOME"
                 value="D:\product\odiclient\oracledi" append="false"/>
    </environment>
    <process-type id="OdiAgent" module-id="CUSTOM">
       <process-set id="OdiAgent" restart-on-death="true" numprocs="1">
          <module-data>
             <category id="start-parameters">
                <data id="start-executable" value="D:\product\odiclient\oracledi\bin\agent.bat"/>
                <data id="start-args"
                      value="-PORT=20900 -NAME=MYAGENT"/>
             </category>
             <category id="stop-parameters">
                <data id="stop-executable"
                      value="D:\product\odiclient\oracledi\bin\agentstop.bat"/>
                <data id="stop-args" value="-PORT=20900"/>
             </category>
          </module-data>
       </process-set>
    </process-type>
</ias-component>
...

 

Starting the ODI agent is done by:

opmnctl startproc process-type=OdiAgent

 

Stopping the ODI agent is done by:

opmnctl stopproc process-type=OdiAgent

 

After restarting or reloading the opmn you can see ODI agent starting up and showing up in the status list:

D:\product\10.1.3.1\OracleAS_1\opmn\bin>opmnctl status

Processes in Instance: soasuite.hkaukovu-fi
---------------------------------+--------------------+---------+---------
ias-component                    | process-type       |     pid | status
---------------------------------+--------------------+---------+---------
OdiAgent                         | OdiAgent           |   20780 | Alive
ASG                              | ASG                |     N/A | Down
OC4JGroup:default_group          | OC4J:home          |   31364 | Alive

Switching BPEL Process File Based Decision Service to WebDAV Based

You might get into a situation where you have started the BPEL Process development using file based business rule repository and noticed later on that you should have been using WebDAV instead.

 

Once you have defined the file based business rule service in BPEL process, the file repository is actually copied under the deployment structure:

<BPEL process>/decisionservices/<servicename>war/WEB-INF/repository

Please note that making any changes on the original rule file will not have any effect on the rule repository deployed and copied with the BPEL process. This is why using WebDAV repository has it's clear role as central place for business rule repository.

 

How to change file repository to WebDAV?

It is possible to replace the file based repository definition to use WebDAV based repository, with surgical operation on BPEL decision service deployment files.

 

0. Make a backup of the BPEL process you intend to modify.

1. Make sure you have the latest business rule repository imported in the WebDAV location. If you don't have that, you can just connect to WebDAV repository from the rule author application and press "Import" to import the file repository file as such. This will extract the file repository contents into WebDAV file structure. Do not copy the file repository "as is" into WebDAV folder as this will not work. You need to import it using rule author.

 

2. (Optional) Create an empty BPEL process and drag&drop a decision service to the process. Define a decision service to use the wanted WebDAV rules connection. Please note that this step is only done to get the needed XML fragment from decisionservices.decs -file.

<repository type="WebDAV">
  <webdav>
    <url>http://myhost:7777/rule_repository/poc/</url>
    <username>harri</username>
    <password encrypted="true">dsIJkj9898dh</password>
  </webdav>
</repository>

3. Open up the actual BPEL process project. Navigate to the file system level to see the decision service deployment directory that was created behind the scenes the first time you defined the decision service using file based repository.

 

Look for decisionservices.xml under:

<BPEL process directory>\decisionservices\<servicename>\war\WEB-INF\classes

 

The file has the configuration info for the decision service, among others something like this:

<repository type="File">
  <file>repositoryresource:MyRuleSets</file>
</repository>

Replace that with (extracted from an existing decisionservices.decs -file from step 2):

<repository type="WebDAV">
  <webdav>
    <url>http://myhost:7777/rule_repository/poc/</url>
    <username>harri</username>
    <password encrypted="true">dsIJkj9898dh</password>
  </webdav>
</repository>

... and save.

 

What I did was also removed the rules repository from:

<BPEL process directory>\decisionservices\<servicename>\war\WEB-INF\repository

just to make sure the service is not using the file based rule repository any more.

 

4. Redeploy the BPEL process.

5. Test.

You should now be using the WebDAV based rules repository with your BPEL decision services.

New syntax for Oracle WSM 10.1.3.3 wsmadmin md5encode

For those that might have seen OWSM md5 hash encoding utility (e.g. for .htpasswd file), there has been a slight change in the syntax and parameters for 10.1.3.3:

wsmadmin md5encode <file> <username>

the old syntax (10.1.3.1) was instructed like:

wsmadmin md5encode <username> <passwd> <file>

Latter doesn't work in WSM 10.1.3.3 any more.

Axis2 Web Services Container on OC4J

I was working on a project to call Oracle Data Integrator (ODI) from Oracle ESB as part of integration scenario. Part of the ODI installation is to set up infrastructure for public web services, that e.g. allows to call ODI integration scenarios remotely using Web Services calls. ODI Public Web Services uses Apache Axis2 as the web services container.

Installing Axis2 on OC4J 10.1.3. was a real "as easy as 1-2-3" task. Downloaded the WAR Distribution zip file from http://ws.apache.org/axis2/download.cgi, unzipped it on my hard disk and then deployed it using Oracle Application Server control.

After Axis2 WAR deployment, the web services container was ready to rock and to upload the odi-public-ws.aar file required by ODI.

This is just as a reminder for people that wonder how to work with Axis2  and OracleAS.

Unconventional Oracle Installation

Some competitors claim Oracle DB installation takes many days and lots of resources.

These Danish hard core Oracle professionals prove this is not true:

http://www.youtube.com/watch?v=CHzV4LZnvHc

 

Please don't try this at home, at least alone ;)