Coach Thrasher

Parkinson's Law of Triviality (a.k.a. the bikeshed)

With yesterday's successful release under our belts, I've re-started thinking about features for the next release. There are a lot of things queued up and waiting to be tested in our product, by our customers. What takes priority?

I found this to be an interesting prequel to evaluation:
http://en.wikipedia.org/wiki/Parkinson's_Law_of_Triviality

The point being, "Should we build it?" may need to become: "How do we test it?" The outcome of which, to me, is that the easiest things to test, at a product level, should be done first. What are the easiest things that we can get in front of customers and get feedback on now?

Posted at 12:16PM Mar 06, 2010 by jason in Software  |  Comments[0]

Don't update OSX: Java For MacOSX 10.5 Update 6 breaks WSS4J

After letting Apple's software update run, I'm no longer able to get WSS4J to read my PKCS12 or JKS key stores to allow for SSL calls using CXF. Crap!

There aren't any other reports of this out there yet (the update went live Dec 3, 2009), that I can find, so I'm posting in the hope that anyone else is having this issue will provide details of the problem, or resolution. Their official documentation for reversion of the Java Update 6 is to reinstall the OS.

Java for Mac OS X 10.5 Update 6
Security update details

I found the problem because my CXF stack is being used to talk to Amazon Web Wervices, using WSS4J. Here's the error I'm seeing after applying Apple's Java Update (below). Note that the "Keystore was tampered with, or password was incorrect" message is misleading as I've just create the keystore, and verified that it's valid with the right password. The problem seems to be that "java.security.KeyStore" isn't seeing the credentials that the Apache WSS4J package is passing in. Since WSS4J hasn't changed, and it worked before the VM update, it looks like a VM bug.

WARN [main] PhaseInterceptorChain.doLog(361) | Interceptor has thrown exception, unwinding now

java.lang.RuntimeException: org.apache.ws.security.components.crypto.Merlin cannot create instance

	at org.apache.ws.security.components.crypto.CryptoFactory.loadClass(CryptoFactory.java:225)

	at org.apache.ws.security.components.crypto.CryptoFactory.loadClass(CryptoFactory.java:180)

	at org.apache.ws.security.components.crypto.CryptoFactory.getInstance(CryptoFactory.java:73)

	at org.apache.cxf.ws.security.wss4j.AbstractWSS4JInterceptor.loadSignatureCrypto(AbstractWSS4JInterceptor.java:195)



[SNIP]


Caused by: java.lang.reflect.InvocationTargetException

	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)

	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)

	at java.lang.reflect.Constructor.newInstance(Constructor.java:501)

	at org.apache.ws.security.components.crypto.CryptoFactory.loadClass(CryptoFactory.java:211)

	... 38 more

Caused by: org.apache.ws.security.components.crypto.CredentialException: Failed to load credentials.

	at org.apache.ws.security.components.crypto.AbstractCrypto.load(AbstractCrypto.java:174)

	at org.apache.ws.security.components.crypto.AbstractCrypto.(AbstractCrypto.java:135)

	at org.apache.ws.security.components.crypto.Merlin.(Merlin.java:71)

	... 43 more

Caused by: java.io.IOException: Keystore was tampered with, or password was incorrect

	at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:768)

	at java.security.KeyStore.load(KeyStore.java:1150)

	at org.apache.ws.security.components.crypto.AbstractCrypto.load(AbstractCrypto.java:168)

	... 45 more

Update: I've resolved it by using my Time Machine backup to revert the entire subdirectory of: /System/Library/Frameworks/JavaVM.framework" and it has fixed the problem. Though Apple's official line is that an entire OSX reinstall is needed to revert, this has indeed reverted both JDK 1.6 and 1.5 correctly. The Software Update app doesn't think that Java Update 6 has been applied.

Posted at 11:44AM Dec 10, 2009 by jason in Software  |  Comments[4]

Auto-generating Maven Exclusions

Identifying maven exclusions for project dependencies can be a tedious pain in the neck of hand coding XML in the pom. My philosophy on XML is that it was not designed for hand editing: so be lazy and don't do it!... or at least try not to.

Here's my little grep/sed recipe to generate exclude statements for all top-tier dependencies of a module. I take its output and paste it into my own modules pom.xml to remove everything, except exactly what I need.

mvn dependency\:tree \

 | grep "\] +- .*compile$" \

 | sed "s/^\[INFO\] +- \([a-zA-Z0-9\._-]*\):\([a-zA-Z0-9\._-]*\):.*$/<exclusion><groupId>\1<\/groupId><artifactId>\2<\/artifactId><\/exclusion>/g" \

 | sort

<exclusion><groupId>com.adobe.acrobat</groupId><artifactId>acrobat</artifactId></exclusion>

<exclusion><groupId>commons-httpclient</groupId><artifactId>commons-httpclient</artifactId></exclusion>

<exclusion><groupId>commons-lang</groupId><artifactId>commons-lang</artifactId></exclusion>

<exclusion><groupId>javax.jcr</groupId><artifactId>jcr</artifactId></exclusion>

<exclusion><groupId>javax.mail</groupId><artifactId>mail</artifactId></exclusion>

<exclusion><groupId>log4j</groupId><artifactId>log4j</artifactId></exclusion>

<exclusion><groupId>org.apache.jackrabbit</groupId><artifactId>jackrabbit-core</artifactId></exclusion>

<exclusion><groupId>org.apache.jackrabbit</groupId><artifactId>jackrabbit-ocm</artifactId></exclusion>

<exclusion><groupId>org.apache.lucene</groupId><artifactId>lucene-core</artifactId></exclusion>

<exclusion><groupId>org.apache.ws.security</groupId><artifactId>wss4j</artifactId></exclusion>

<exclusion><groupId>org.hibernate</groupId><artifactId>hibernate</artifactId></exclusion>

<exclusion><groupId>org.springframework</groupId><artifactId>spring-core</artifactId></exclusion>

<exclusion><groupId>org.springframework</groupId><artifactId>spring</artifactId></exclusion>

<exclusion><groupId>org.springmodules.jcr</groupId><artifactId>springmodules-jcr</artifactId></exclusion>

<exclusion><groupId>rome</groupId><artifactId>itunes-com-podcast</artifactId></exclusion>

<exclusion><groupId>rome</groupId><artifactId>rome</artifactId></exclusion>

<exclusion><groupId>xalan</groupId><artifactId>xalan</artifactId></exclusion>

Posted at 11:43AM Oct 22, 2009 by jason in Software  | 

37signals Valuation Tops $100 Billion

Here Joel Spolsky's "Duct Tape Programmer," piece, re-written by Jason Fried of 37signals:

37SIGNALS VALUATION TOPS $100 BILLION AFTER BOLD VC INVESTMENT.

The next time I hear about someone seriously pitch me on creating a world-changing app in 30 minutes using GUI tools, I'm going to vomit.

Posted at 09:14AM Sep 25, 2009 by jason in Software  | 

Feeling like MacGyver

This had me feeling like MacGyver:

The Duct Tape Programmer

When the shit hits the fan, don't get sprayed by a complex system. But sometimes it's unavoidable, damn.

Posted at 01:57PM Sep 24, 2009 by jason in Software  | 

rtmpdump

Wow, Luke Kenneth Casson Leighton is pissed at Adobe for DMCAing RTMPdump. I believe in free software, and not breaking copyrights, so I side with him on those points. However, when he brings his 7-week old daughter into the argument with an f-u sign, that crosses the line. Bad form Luke, baaaaaaad.

Posted at 01:05PM May 25, 2009 by jason in Software  |  Comments[1]

GMail via Spring JavaMailSenderImpl

Sometimes I hate Google. It can give too many wrong answers. Yet, they look so enticing, like I'm a little kitten playing for hours with a ball of yarn. It gets me nothing.

None the less, here's how to properly configure Spring's JavaMailSenderImpl to -properly- send mail through Google Mail using your gmail account.

      <bean id="mailSender" class="org.springframework.mail.javamail.JavaMailSenderImpl">

          <property name="defaultEncoding" value="UTF-8"/> 

          <property name="host" value="smtp.gmail.com"/>

          <property name="port" value="465"/> 

          <property name="username" value="${mail.username}"/>

          <property name="password" value="${mail.password}"/>

          <property name="javaMailProperties">

              <value>

              	mail.debug=true

              	mail.smtp.auth=true

              	mail.smtp.socketFactory.class=java.net.SocketFactory

              	mail.smtp.socketFactory.fallback=false

              </value>

          </property>

      </bean>

Posted at 04:55PM Mar 31, 2009 by jason in Software  | 

SVN diff with Meld

Every time I setup a new machine for development I forget a whole bunch of extra little bits that make developing more efficient and enjoyable. Most recently a new Macbook Pro has been my platform of choice. I like the visual diff implemented by Meld, and it works nicely using X11 with the Meld build coming from Fink. (Don't even try installing Meld from MacPorts... after 24 hours of hardcore Python pain it totally failed.) Here's a little script to launch a visual diff against the SVN repository that I keep misplacing (captured on my blog so I can find it again).

svn-diff-meld

1) download it
2) move it to ~/bin/svn-diff-meld
3) chmod +x ~/bin/svn-diff-meld
4) echo "alias sd='svn diff --diff-cmd ~/bin/svn-diff-meld'" >> ~/.bash_profile

Then on the command line, run SVN diffs against the repo with something like:

$ sd -r 27003

or

$ sd path/to/some/file.txt

or

$ sd -r 27003 path/to/some/file.txt

Posted at 09:21AM Aug 15, 2008 by jason in Software  | 

RegCM and the Scientific Method

My wife is doing her graduate work at UCSC and using the RegCM regional climate model to simulate atmospheric conditions during the Eocene era. She frequently finds bugs in the code (FORTRAN), but she has no way to keep track of what changed other than creating a versioned tar ball of the whole code tree for each version change.

Recently she found a bug that affects all of the research she's done for the past 3 years. The bug likely affects countless other researchers and their published work, possibly invalidating their findings. This sounds incredibly troubling to me, but there's no way to understand what bugs those other researchers may have been affected by because they don't reference a version, branch, or tag, of any code tree.

How can any of their research be repeatable if they don't reference a version of their code? This seems like a fundamental flaw in implementation of the scientific method.

This has been driving me crazy because I would never embark on a large software project without a source control system. I would go insane tracking all of the changes, and trying to ascertain what happened when something didn't work.

After a little searching, I found a free SVN hosting solution at Code Spaces for her to use. It's not hosted at SourceForge, Google Code, or other OSS access points because we have little understanding of the licensing controls on the code, which is managed by some nice Italian climate researchers.

If anyone is interested in using the SVN system to track changes, here's where it lives:

http://svn.codespaces.com/regcm/regcm/

To check it out, just run:

svn co http://svn.codespaces.com/regcm/regcm/trunk regcm

Posted at 12:30PM Jun 27, 2008 by jason in Software  | 

TTL of Maven Repositories?

While building Spring Security 2.0.2 from their SVN tag I ran into what is increasingly becoming a major Maven annoyance. Namely, access to or maintenance of public Maven repositories is becoming problematic. They don't seem to be maintained, and when they are broken, many OSS projects aren't fixing references to them.

Spring Security, on the 2.0.2 SVN tag, won't build because it can't find the appropriate JetS3t. This is because the JetS3t jar doesn't exist where it once did. While I had no idea where it once lived, even the JetS3t Downloads page has incorrect instructions to find it: the referenced Maven repository URL doesn't seem to reference a valid S3 bucket. Maybe someone stopped paying the S3 bills?

So, while I was trying to update my JCaptcha code for AppFuse, I couldn't build the main Spring Security tree, nor the sandbox/captcha module required for JCaptcha. What a pain in the butt. Now I had to hunt down the correct version of the JetS3t jar to get Spring Security to build.

It turns out there's an interesting S3 based storage service called S3Browse that links to what appears to be a mirror of the former JetS3t maven repo path here: http://s3browse.com/explore/maven.springframework.org/external/. It's a bit shady because the jar isn't coming from the JetS3t file's owner, and I have no idea what's compiled into the version sitting on S3browse. Considering S3Browse's homepage doesn't seem to have been updated since 2007, I'm skeptical the link will live long.

So that begs the question: whats the time to live of OSS Maven repositories? I have faith Apache's will stick around, but there seems to be an opportunity for a repo-aggregator here to suck up the slack.

Posted at 10:43PM Jun 12, 2008 by jason in Software  | 

Glass Door

I just read about Glass Door, which shows salaries of companies that I'm competing against in the hiring relm. Specifically, we're competing against Google and Yahoo to hire the top talent in the Bay Area.

I'm very happy to say that 4INFO pays at the same level across the board. We need the brightest people, and must show that we'll compensate them appropriately in order to hire. It turns out that we're doing just that. I couldn't be prouder of Software Engineering at 4INFO!

Posted at 06:27PM Jun 12, 2008 by jason in Software  | 

Parameterized user.home for Maven on Windows XP

I found a handy way to reference the Java system property ${user.home} within a Maven build that supports Windows' ridiculous path name to home/profile directories:

c:\Documents and Settings\jason.

The problem is, when using Maven, this parameterized property doesn't get passed through as one property value, but as three, because somewhere in the build Maven chokes on the spaces or back-slashes and interprets it as either three arguments:

"c:\Documents", "and", "Settings\jason"

or treats the windows back-slash as an escape character and removes them so my parameterized user.home becomes:

"c:Documents and Settingsjason"

Why would this matter? Well, if you want to share build configurations with other developers, and can't check some of your files into a source control management system (because, for example, they contain private keys or certificates that don't belong in SCM), then you need a common, parameterized place to reference them that will work for every developer without much trouble. In my case, it's integration with Amazon's web-services that necessitates managing public/private PEM and P12 files.

For Amazon, I want to reference the AWS pem and p12 files like this from either the Maven pom.xml, or the settings.xml file:
${user.home}/aws/cert-BLAH.pem
${user.home}/aws/pk-BLAH.pem
${user.home}/aws/awskeystore-BLAH.p12

However, on Windows XP, unless I set the user.home on the build path every time, the back-slash escaping or space issues cause the files to not be found.

To fix it, add this profile to the $M2_HOME/conf/settings.xml file:

<profile>
<id>laptop-xp</id>
<properties>
<user.home>C:/Documents and Settings/${user.name}</user.home>
</properties>
</profile>


Then add an appropriate entry to the activeProfiles:

<activeProfile>laptop-xp</activeProfile>


Now every user will be able to use the user.home property to reference their home path correctly on the Windows box.

Posted at 10:13AM Jun 08, 2008 by jason in Software  | 

Mac OSX Terminal Colors

By terminal colors, I don't mean the colors of death, I do mean the command prompt. This dude has a great post on how to enable terminal colors on OSX. It's quite useful for reducing fatigue on the eyes (black background), and syntax highlighting within vi for different file extensions. My eyes get lost in the black and white of XML or Java after a few minutes on the stock terminal prompt.

Posted at 03:30PM Jun 02, 2008 by jason in Software  | 

Facebook API Better Stronger Faster

Okay, I've released my "better stronger faster" Facebook API on Sourceforge for all to use. It’s code name is F8-API, and it’s available for all to download or browse from the SVN repository and try out.

A download package hasn't been released yet, as I'd rather people provide feedback to validate that a release is warranted before packaging it for distribution. Use this command to get the files from Subversion:

svn co https://f8api.svn.sourceforge.net/svnroot/f8api f8api

I had signed up as a "Facebook developer" in several of the Facebook apps tools, and received about 1 query a day from people looking to "build the next killer app". Unfortunately, I found that people tended to make the assumption that I was a college student with lots of time on my hands to do this, and I'd work for peanuts. Well, that's not the case, but I do reflect fondly on college, and I can afford my own peanuts, thank you. As I result, I decided to give away some of this love on Sourceforge, and such, the project was born.

Today the F8-API is a lightweight stateless interface to Facebook’s web services. It uses Apache HttpClient, and Log4j, and it processes data quickly through SAX parsers. It is completely functional for all API calls today, and it supports both desktop and web based Facebook applications. You can also write unit tests that will hit the live Facebook account and interact with your test environment using Maven and JUnit, for both desktop and web apps.

Hopefully people will find this useful as it is. There is some room for improvement though. The data model used by Facebook needs to be flushed out so a "higher level" API can be interfaced that will act on model objects. As an applications developer, you usually don’t want to fiddle with low level things like the transport layer of an API that you’re interfacing. So, some flushing out is needed here.

I also need to write some example code to show how easy it is to use. If you’re subscribed to my RSS feed, you’ll see the update notice posted to the feed soon.

Posted at 10:44AM Sep 13, 2007 by jason in Software  | 

YAY, I Fixed my Facebook API :)

Like everyone and their little brother, I've been working on a killer Facebook app. The platform offers so much potential for revenue generation; I just can't pass the opportunity up. I found quite a few architecture issues with the Java Facebook API that Facebook released. It worked, but it wasn't enterprise ready... in particular there were no JUnit tests, no logging, HTTP connection memory leaks, it’s not thread safe, etc. It works for "Hello World", but not my app.

The Javabook API looks cool, but I needed desktop support. I also didn't want to use JSON since FQL returns buggy (ambiguous) results that way for queries (see the Facebook docs, I do like the name though, “JSON” == great name :).

Aside: what’s the business goal of releasing a 3rd party Facebook API? I don’t think the answer is anything close to the logic that created Apache.

So, I've finally fixed my API bug. I don't mean to beat-up on Facebook (because their site, app, business, and developers absolutely rock), but the issue was related to their API documentation. The references to "session", "session key", "session secret" and "secret" for desktop apps versus webapps are ambiguous in their API docs. They are used on different pages in different context to mean similar and different things, depending on the page. Ack! Good documentation is critically important for communicating with others, and this issue just highlighed that for me.

I posted a Mavenized Facebook api (the Facebook original code plus a few JUnit tests plus Maven wrapping) a few days ago. It's been interesting to see how many downloads per day there are... not many == ~4. But heck, there are downloads! So there must be demand for something like this.

If anyone is interested in a more robust API for desktop and webapps, using HttpClient with logging, and per-UserId sessions, thread safety, local callback support, and Maven let me know. It supports all Facebook API calls, including FQL. The code is fully documented with Javadoc. It'll do XML and JSON. It supports Spring-based configuration out of the box. It has a Facebook data model, with MVC style stateless API calls. It'll put your API key and secret through their paces for all API calls using JUnit. It's so good, it'll get you a beer when asked. I’ve got it humming right here and would be happy to license it out.

Posted at 11:14PM Aug 07, 2007 by jason in Software  |  Comments[1]