Birds, Frogs and � Brogs?

By Esther Soloveichik

(Back to article)

I have always been intrigued by physicists who study things they cannot see. They study the atom, for example, by observing the visible turbulence left in its wake. It’s like deducing a boat has passed while watching the water.

Business intelligence (BI) is not physics, but there is at least one similarity in the disciplines. BI practitioners look for the stories in the data by studying the visible tracks left in the muddle of source systems.

Browsing the Web recently, I was struck by the opening lines of an article on Freeman Dyson, the “frog prince of physics” and its import on business intelligence.

Freeman Dyson loves the metaphor that divides scientists into two groups: Birds, who look down upon everything and have a God’s-eye view of the world, and frogs, who spend their time in the mud. The renowned Princeton physicist calls himself a frog.

“I’m not against the first group, but they take an exalted view of science. Frogs typically enjoy exploring things locally and developing skills,” said Dyson in an 1999 Salon.com article by Kristi Coale.

With remarkable clarity, I saw immediately what most data warehouse projects lacked brogs—amalgams of birds and frogs.

Best of Both Worlds

Obviously, I am talking about people here. A brog lives in two worlds and segues seamlessly between them: the bird’s view of business objectives, and the frog’s mud-plowing efforts to make those objectives a reality.

A data warehouse project staffed by a small team of brogs, helping with, or taking on multiple roles within the data warehouse lifecycle, has a better chance of succeeding than a project staffed with disparate people, each playing a singular bird- or frog-like role of project manager, business analyst, data modeler, DBA, ETL designer, ETL developer, end user application designer, and developer.

The ability to flit between business needs and data minutiae is finely honed. It comes from a broad and deep understanding of both transaction processing and the full data warehouse lifecycle. Although the methodology for designing and developing a data warehouse is quite different from the one used in a transactional application, it is the accumulated memory of both worlds that forge a brog.

An experienced trekker of business transactions understands the trip down the business lifecycle and its many legs. She can recognize the questions the business wants answered, and where the answers can be found.

Likewise, the experience of acquiring, cleaning, and moving the right data into a data warehouse within ever-narrowing timeframes creates a keen awareness of the need for a sensibly scoped undertaking, when and how to deal with dirty data, and the value of navigable and responsive query applications.

It is this accumulated memory that gives a brog the peculiar ability to see things not readily visible to others. She can visualize the patterns of user queries, the dimensional models needed to facilitate those queries and the constraints imposed by the realities of the data. Every data warehouse project has aspects that are new and untried, and slip-ups are natural. A penchant for burrowing in the mud unearths subtle problems near the beginning of the project. Identifying wrong solutions sooner, coupled with an aversion for obfuscation, leads to timely corrective action and saves much heartache later on.

Follow Your Nose

Like the physicist who studies the tracks left by the atom, the brog has a feel for what trails to follow and what data to stalk, iteratively gaining knowledge to further hone in on desired footprints. The business objectives are adjusted by the subtleties in the data, and the search for the right data is continually influenced by the business objectives.

Staffing an entire data warehouse project with brogs may be difficult. At a minimum, the project must be scoped appropriately and the data modeled correctly. That means that the lead business analyst—preferably, cum data modeler—should be a brog.

The analyst maintains the bird’s view by understanding the business objectives of the organization and elicits the questions the business wants answered to help achieve those objectives.

At the same time, with the help of the ETL designer, she digs into the source systems to discern the existence, accessibility and quality of the data needed to answer those questions. She keeps digging until reasonably confident of the feasibility of migrating the source data to support the business requirements.

A brog is ever practical. She starts with a data warehouse that does not do everything, but one that can mature to adulthood. Since she cannot foresee all that the data warehouse may face downstream, the analyst/modeler ensures the viability and extensibility of the warehouse by adhering to good data warehouse design and by balancing the exalted view of the business with the reality of mud-splattered data.

What secondary roles ought this analyst brog take on?

She can assist with ETL design. The ETL design and development phase of a data warehouse project consumes most of the effort and is burdened with most of the risks. Including the ETL designer in requirements gathering and the data audit, and actively drawing the analyst into the data staging effort, reduces errors and ramp-up time inherent in transition handoffs.

The analyst brog can also help specify an initial set of end-user report templates. She understands that, to the business users, the reports are the data warehouse. Regardless of the wizardry displayed in migrating the data, if the reporting application is difficult to use, or the response time is slow, the data warehouse has failed.

The analyst might also assist with data stewardship, protecting and assuring the integrity of the data as it migrates from the source systems to the data warehouse.

Where does one find a brog? Does this paragon exist?

Look for someone with an accumulated memory of transactional and dimensional systems. It is rare to find a person who has performed every role in both the transaction and data warehouse lifecycles, but it is possible to find someone who was actively involved in many aspects of both worlds.

And if you think you are a brog, try putting it on your resume. If nothing else, it should make for interesting conversation.

Esther Soloveichik is senior consultant with Intrasphere Technologies, a technology consulting firm with a core focus on life sciences. Intrasphere provides end-to-end technology services and has successfully implemented large-scale projects for some of the world’s leading global companies, including Pfizer, Schering-Plough, Novartis and Eli Lilly, among others.