The back story is one I have purposefully repeated whenever giving a presentation about the project, namely, that our Pika Plan A did not work. We encountered code anomalies in Pika that, among other things, cause it to auto-generate new case intakes and case records when it is crawled by the GSA. As a result, we were unable to use the GSA to crawl the Pika client case content dynamically generated as web pages. Plan A would have been the easiest, no-brainer way to go but we were not able to do so. So Plan B was to have the GSA target the Pika MySQL database directly. Status report: Mission accomplished.

There are GSA capacity issues for us, since our particular GSA’s one million “record” capacity means one million web pages or database records, inclusive, and these database records are not the same thing as the count for client case records. At any given time, we may have some 130,000 to nearly 200,000 client cases in our Pika system (and even more in archival data storage), but from a database perspective, these add up to multi-millions of “records,” e.g., various types of time records, case notes, contacts, and so on. Part of the challenge for us was to sort out which pieces of those millions of database records were the ones most needed and useful to our users.

The solution? Using a well-tailored query, we have the GSA do a selective crawl of the Pika MySQL database to return the most commonly sought and used Pika content: Case numbers, client names, office designations and case notes… tons of case notes. The basic technical explanation is the GSA performs a database query, returns it as an XML feed, indexes that feed, against which the user’s search terms are queried and ultimately returned as viewable HTML

What does the the search result look like? A Google search result. The clickable link displays the case number, client name, LSNC office and primary advocate name, e.g., “90-10-123456 ~ John Client ~ Sacramento ~ Jane Advocate.” Below that it displays in-context text with the search terms highlighted in bold, essentially like a regular Google search result. Clicking the link dynamically displays the actual Pika case note shown in context. Assuming there are multiple possible matches for a particular Pika case record, there is a link to display all the “omitted results,” akin to how regular Google searches work, so the users can see all possible, not just probable matches. Clicking through the GSA search result link also gives the user direct clickable access to the particular client case record since clicking through takes the user to the actual Pika client case record.

That’s the name of that tune.

Yes, that’s right. Just (mouse) click our Pika home page three times and you are magically transported back home.

The infusion of new, future content into the simplified structural taxonomy we created is a separate challenge we will be posting about later. Dealing with our existing files is more immediate, more concrete. Groking those files has been one of the more interesting, at times hilarious parts of this project.

For those legal services programs interested in how the existing files from our eight local offices break out, here are the percentages for the seven most common “document” types:

• 67% – WordPerfect (WPD)
• 18% – Word (DOC)
• 9% – Portable Document Format (PDF)
• 3% – Excel (XLS)
• 1% – Text (TXT)
• 0.9% – Rich Text Format (RTF)
• 0.6% – PowerPoint (PPT)

(Discretion prohibits us from detailing the other file flotsam discovered on local office servers. That said, allow us to observe that some within our organization have extraordinarily good taste in photos taken by National Geographic, and not such good taste in music.)

We are totally on track for targeting most of our planned GSA targets: The existing office archive files listed above have long been targeted (although we still have a lot of work left to fit them into our structural taxonomy); over the last several months we have worked very hard to refresh and update (and remove, as warranted) the targeted content at LSNC’s various public websites; and we are very pleased with the quality of the GSA results we are getting out of Google Sites.

This is all good news. In addition, we are putting in place a few more content channels: Targeting the content in our organization’s seven private Google discussion groups, and a program-wide canvass for select hard-copy training resource materials for digital conversion and addition to the shared repository. It’s all good.

We have had one major disappointment: We discovered that there are significant, unanticipated technical challenges unique to the Pika CMS that thus far have prevented effective use of the GSA to target Pika content. The problem is not the GSA itself or configuring the GSA to target Pika. The GSA by design performs wholly benign, non-destructive crawls as it indexes targeted records. We did a huge amount of target testing and SERP evaluation, and we were very pleased — actually, thrilled is a better word — with the results we were getting from Pika. The unanticipated problem is that the current version of Pika is not well optimized for use as an enterprise search target. There are code anomalies in Pika that, among other things, cause it to auto-generate new case intakes and case records when it is crawled by the GSA.

After an assessment by Pika Software, it is now apparent it will take something in the neighborhood of 200 hours of work to make Pika more receptive, shall we say, to an external crawl by the GSA. (Ouch!) So for now, we have to put that part of the project to the side. File under: Lessons Learned.