For those who have followed The Findability Project, I am pleased to report we have surmounted the basic technical problems of targeting our Pika CMS with the Google Search Appliance.

The back story is one I have purposefully repeated whenever giving a presentation about the project, namely, that our Pika Plan A did not work. We encountered code anomalies in Pika that, among other things, cause it to auto-generate new case intakes and case records when it is crawled by the GSA. As a result, we were unable to use the GSA to crawl the Pika client case content dynamically generated as web pages. Plan A would have been the easiest, no-brainer way to go but we were not able to do so. So Plan B was to have the GSA target the Pika MySQL database directly. Status report: Mission accomplished.

There are GSA capacity issues for us, since our particular GSA’s one million “record” capacity means one million web pages or database records, inclusive, and these database records are not the same thing as the count for client case records. At any given time, we may have some 130,000 to nearly 200,000 client cases in our Pika system (and even more in archival data storage), but from a database perspective, these add up to multi-millions of “records,” e.g., various types of time records, case notes, contacts, and so on. Part of the challenge for us was to sort out which pieces of those millions of database records were the ones most needed and useful to our users.

The solution? Using a well-tailored query, we have the GSA do a selective crawl of the Pika MySQL database to return the most commonly sought and used Pika content: Case numbers, client names, office designations and case notes… tons of case notes. The basic technical explanation is the GSA performs a database query, returns it as an XML feed, indexes that feed, against which the user’s search terms are queried and ultimately returned as viewable HTML

What does the the search result look like? A Google search result. The clickable link displays the case number, client name, LSNC office and primary advocate name, e.g., “90-10-123456 ~ John Client ~ Sacramento ~ Jane Advocate.” Below that it displays in-context text with the search terms highlighted in bold, essentially like a regular Google search result. Clicking the link dynamically displays the actual Pika case note shown in context. Assuming there are multiple possible matches for a particular Pika case record, there is a link to display all the “omitted results,” akin to how regular Google searches work, so the users can see all possible, not just probable matches. Clicking through the GSA search result link also gives the user direct clickable access to the particular client case record since clicking through takes the user to the actual Pika client case record.

That’s the name of that tune.

As promised, here is a set of 15 representative screenshots of Legal Services of Northern California’s most recent customization of Pika 4.0. To view a larger image, right click on it and open the image in a separate window. (Needless to say, none of the data field entries in the screenshots are real.) It shares obvious design elements with LSNC’s shared portal, which serves the “news and resource” purpose of the original Pika home page.

Yes, that’s right. Just (mouse) click our Pika home page three times and you are magically transported back home.

Many months of fish to fry ahead, including bearing down on LSNC’s customized rebuild of Pika 4.0 and working out a solution for integrating our Google Search Appliance. So, laying low until October 1. See you back here this Fall and in early 2010 at the next Austin TIG, when we serve up the whole enchilada.

There is a great deal of work proceeding behind the scenes and several key project elements are converging as we move toward finalizing this public project. Among other things, we have been working through modest but practical solutions for better placement and targeting of our existing 300,000+ repository documents, while solidifying all the additional Google Search Appliance (GSA) targets in our enterprise search sights, described in Part Two. At the same time, we are in our own March (and April and May) Madness as we mount a rapid-fire round of trainings for each of our eight remote offices (spread out over 50,000 square miles of Northern California) on their new role in making Google Enterprise search work for them, which is to say for all of us. And as mentioned in earlier posts, we are working every bit as earnestly on our latest in-house build of the Pika CMS.

The infusion of new, future content into the simplified structural taxonomy we created is a separate challenge we will be posting about later. Dealing with our existing files is more immediate, more concrete. Groking those files has been one of the more interesting, at times hilarious parts of this project.

For those legal services programs interested in how the existing files from our eight local offices break out, here are the percentages for the seven most common “document” types:

• 67% – WordPerfect (WPD)
• 18% – Word (DOC)
• 9% – Portable Document Format (PDF)
• 3% – Excel (XLS)
• 1% – Text (TXT)
• 0.9% – Rich Text Format (RTF)
• 0.6% – PowerPoint (PPT)

(Discretion prohibits us from detailing the other file flotsam discovered on local office servers. That said, allow us to observe that some within our organization have extraordinarily good taste in photos taken by National Geographic, and not such good taste in music.)

We are totally on track for targeting most of our planned GSA targets: The existing office archive files listed above have long been targeted (although we still have a lot of work left to fit them into our structural taxonomy); over the last several months we have worked very hard to refresh and update (and remove, as warranted) the targeted content at LSNC’s various public websites; and we are very pleased with the quality of the GSA results we are getting out of Google Sites.

This is all good news. In addition, we are putting in place a few more content channels: Targeting the content in our organization’s seven private Google discussion groups, and a program-wide canvass for select hard-copy training resource materials for digital conversion and addition to the shared repository. It’s all good.

We have had one major disappointment: We discovered that there are significant, unanticipated technical challenges unique to the Pika CMS that thus far have prevented effective use of the GSA to target Pika content. The problem is not the GSA itself or configuring the GSA to target Pika. The GSA by design performs wholly benign, non-destructive crawls as it indexes targeted records. We did a huge amount of target testing and SERP evaluation, and we were very pleased — actually, thrilled is a better word — with the results we were getting from Pika. The unanticipated problem is that the current version of Pika is not well optimized for use as an enterprise search target. There are code anomalies in Pika that, among other things, cause it to auto-generate new case intakes and case records when it is crawled by the GSA.

After an assessment by Pika Software, it is now apparent it will take something in the neighborhood of 200 hours of work to make Pika more receptive, shall we say, to an external crawl by the GSA. (Ouch!) So for now, we have to put that part of the project to the side. File under: Lessons Learned.

This may have dropped below the radar for some Pika shops: Matt Friedlander has posted at PikaDocs his Pika 4.0 Upgrade Guide, a “short walk-through of the process of upgrading your 3.x Pika CMS to the new 4.0 version released at the end of 2008.”

Load your feed readers, people: Aaron Worley, PHP rock star and progenitor of the Pika case management system widely used within the legal services community, has announced the debut of the official Pika Software Blog. And next week he promises to also roll out a new version 4.0 with a slew of changes.

Of course, at LSNC we are excited to see a new major iteration of Pika coming out. So, I use this august occasion now to cry uncle and formally apologize to Aaron about all my years of posturing about how text resizing in Pika is a problem because of how its default CSS works.

Hey, Pika CSS works just fine, and I’m sure it’ll be great in the upcoming version 4.0. And all my posturing about text resizing over the years is for naught. All the major browsers now default to full-page zoom rather than text resizing. Even though text-resizing is still an option in some browsers, I now must also admit I no longer even look for it. I have surrendered to full page zoom.

So, Aaron, I cry uncle and apologize. History has vindicated you.

Long live Pika CMS!

In no particular order I have posted some representative screenshots of Pika Reborn 2008, an intentionally overstated reference to LSNC’s current experiments with a wholesale rebuild of the Pika 3.07 templates and external stylesheets. (The gallery is flash-based; if you right click an image, you can load a full view of the original screen capture into your browser.)

Working on this modest redesign has been a preparatory exercise for our in-progress updating of our existing version 3.04 install, to fold in the Pika features added since that version, plus integrate additional code changes for custom functionality and modules we want to add to our particular Pika martini. We hope to have our final rebuild completed shortly after the first of the year, and it should look something like this new design. The great advantage of taking a few months to do this “rebuild” experiment is that we were able to try, re-try, adopt, abandon, etc., a number of different solutions for how we want Pika to look and feel and function.

As for the redesign illustrated here, the short version is this:

• All the Pika structural markup was rebuilt, from scratch.
• All embedded and inline styles were stripped out of all HTML template and PHP files.
• All tables used in Pika 3.07 for layout and/or positioning of page elements were stripped out.
• Tables were retained solely for displaying tabular data, e.g., displaying case lists, search results and so on.
• All Pika image elements were replaced or eliminated.
• The default Pika external CSS (cascading style sheets) implementation was dropped and a complete rebuild of external styles was undertaken. This new redesign relies on about 50% less CSS code than was used three years ago for the Project Claire design.

Ya gotta love the Pika. (We do.)

There are two web projects at LSNC that are moving toward a point of convergence: The most notable of the moment is the ground work we are doing on managing our knowledge content and making it all “findable” via a Google Search Appliance (GSA), which is being documented at The Findability Project (TFP). The other is an in-progress code updating and design refresh of the Pika case management system we have been using with great satisfaction for the last several years, which in earlier iterations we documented at PikaDocs and more recently (and less extensively) here as part of the great and ongoing Pika~palooza.

One of the great lessons we have learned from the whole Pika experience is that our users really like having a commonly shared page from which to begin their work. To some extent, this appetite for a shared page or portal was meet by the Pika CMS “home” page, which we modified by using WordPress to generate a functional, customizable message or news screen. Back in the day, which is to say a mere three years ago, when we did our first customizations of Pika, that modification meant our using WordPress 1.5, a suitable plugin to dynamically convert the native WordPress RSS feed into HTML post content, and recoding a few Pika files to get WordPress and Pika to cooperate with each other. Fun to do, but it was still real work to get it done.

That was 2005. Three years later we are implementing enterprise search and now very consciously building an organizational shared portal to supplant the Pika home page. Simply put, in our next iteration of Pika, which we hope to have in place early next year, what Pika people know as the home or start page of Pika CMS will go away. The functional utility of the Pika home page will be folded into our emerging shared portal, the point of search and entry to many things, one of which will be Pika. (It will take work to do it, but we also expect to supplant the Pika native search function and substitute a subset of our Google Search Appliance functionality to accomplish the same thing, only better.)

To get feedback, we are constantly tooling with varied options for the shared portal as we test things out with different LSNC offices. For example, we have already modified the temporary TFP search portal we experimented with earlier this week in a presentation to our Sacramento Office. After that event, we reorganized things to emphasize the location of the news feed, moving it from the right side to the left, and created a temporary second search field to heighten user awareness of our “Google Sites” special collections. (In final implementation, all our document collections will be searchable from an integrated single search field, with various options for filtering the search results.)

And what about that LSNC “news” thing on the shared portal page?

In 2005 we had to do some attentive coding to get it to work with the Pika home page. Three years later, all we needed to do was pop the native WordPress 2.6 feed into FeedBurner’s totally freebie Buzzboost feature, click on a button to generate a small bit of javascript, past that script code into the shared portal page, and there you have it — an HTML standards-compliant, fully CSS customizable, full-text news module.

In late December, after posting an earlier version of this post, I realized I was going about this aspect of Project Grace all wrong, so I pulled that original version, a post about “Basic CSS page layout.” There was nothing wrong with the post, but I realized I was drifting toward just repeating myself, essentially a “remastered” series of articles I wrote, quite demonstrably with less knowledge about Pika than I have now, when building Project Claire.

But there were some deeper problems and challenges for me. As I moved ahead with the challenge of doing another CSS makeover of Pika, I was forced to realize (again) that there are really, really severe limits to what one can do with the CSS “presentational” characteristics of Pika because it still labors under a near-decade of seriously dated legacy markup. (In fairness, when we looked at competing CMS options a few years back, all the other developers suffered from this same shortcoming.) While Pika Software did make a serious effort in version 3.06 to clean out embedded and inline styles, with something like a 40-50% uptick in external CSS coding (a good thing), a huge amount of inline styles remain (a bad thing). Pika still relies predominantly on “tables” to control layout and positioning of page elements, which is a good thing for backward browser compatibility and therefore brings a level of presentational predictability in production software. I totally get that. But otherwise it totally sucks.

This is a two-fisted combo that seriously prevents doing much of anything creative with Pika presentational characteristics. Why? Well, inline styles always trump the external style sheet. That’s how CSS works, so your stuck with whatever styles Pika Software embeds inline. And reliance on tables severely constricts the positioning of page elements. Basically, you’re stuck with the Pika default page layouts unless you are prepared to go in and rebuild your templates to conform to more current web standards, something we did with Project Claire, as best as we could at the time.

The other factor that triggered the change in how to approach Project Grace was a meeting Mark Sawyer and I had with some folks from East Bay Community Law Center, who soon will adopt their first CMS ever, namely, Pika 3.06. And I think it is fair to say they’re feeling overwhelmed. (Whew, way understandable to anyone who’s ever been through the process.)

It was during that meeting I got the idea to take the current version of Pika and do simply what I talk about above, i.e., quite literally rebuilding all the Pika templates and subtemplates, regrettably but of necessity still relying on the deeply embedded tables-based layout structures, while removing all the inline styles that remain in the markup. And then modifying minimally the PHP files essential to dynamically generating other specific Pika page markup, principally markup generated dynamically by pika_cms.php and pikaMisc.php. And then creating a totally custom CSS style sheet, all of which works with a default Pika installation. No changes in how Pika works out of the box. No changes in Pika default workflow. Just better, more flexible markup in the templates, some essential tweaks to a few PHP files to get other markup to display correctly, and a fresh coat of paint for the whole shebang via a custom CSS style sheet. And something one could simply upload to the /pika-custom and /css subfolders in one or two fell swoops.

The goal would be to conjure up a simple, clean, low-contrast, uncluttered design refresh. That’s the theory, anyway. So, with Aaron’s recent announcement that Pika 3.07 will be released in another week or so (and Pika 4.0 put off to later this year, at the earliest), the Project Grace approach has changed: Once version 3.07 hits the street, LSNC will set up three Pika 3.07 test beds: first, a virgin reference install; second, an “East Bay” install to build a new set of default templates with a custom CSS style sheet, as described above; and third, a long-form install where we can work on a more radical rebuild to incorporate all latest, way cool Pika Software modifications into all the existing LSNC customizations, something that may take us the better part of six months or so to complete.

What I will be doing here at Webdogs 2.0 is reporting on progress of the second and third installs. Rather than do long-form tutorials, as I have been doing, what I will do is post the rebuilt templates and the custom “East Bay” CSS to PikaDocs in the California Customizations section. These two installs will most definitely be works in progress. The files posted to Pikadocs will inevitably be added, edited, supplemented, and/or deleted depending on what’s happening as the work goes forward. Most likely, I will post select additions to the Project Grace series here to detail distinctive elements of the templates or CSS designs.

Want an early taste? Take a look at the initial set of 21 screenshots of the “East Bay” design now posted to PikaDocs.

This is a repost of an item I added this morning to the PikaDocs wiki. And it has to do with a very unexpected problem we encountered with the recent Pika 3.06 iteration, which implements code changes that unintentionally strip out IE conditional comments from the Pika default.html template. Needless to say, IE conditional comments are a widely adopted, all-but-standardized way to apply IE-only CSS code to control the design or display of web pages. And LSNC has long relied on custom IE-specific CSS code for its Pika redesign. What were CSS hacks like us to do about this dilemma?

Well, sure enough, those PHP-code ninjas over at Pika Software (Matt, to be specific) had the answer:

Open the pl.php file located in the your Pika subdirectory at /app/lib/pl.php. Carefully comment out lines 2055 and 2056 by preceding them with double-forward slashes so that it looks like this:

// $str = str_replace("<!--[", $tpl_prefix, $str);
// $str = str_replace("]-->", $tpl_suffix, $str);

Save the changes and upload the pl.php file to replace it. Now your IE conditional comments code will play nice with Pika 3.06.