As recently as last month, our user interface—the graphical look our users see every day—was OK. But not better than that. And truth to tell (after all, Intellectual Integrity is a company value), occasionally worse. Yet that “OK” user interface sits on top of the greatest Big Data engine the world has ever known. It sits on an engine that lets thousands of users with little or no formal training reach out every day into mind-numbingly vast collections of billions of documents spread all over the world.
So, can we agree that “OK” isn’t good enough?
I’m not talking about making the product “look pretty.” Sure, I was known to mention “gorgeous” in team discussions of our goal, but there was no way this beauty could be only skin deep.
Our goal is wildly more ambitious: Because we can reach and analyze more information in more formats than anyone else, we need to invent the future of how people find meaning in the current and future ocean of information.
I apologize if this sounds grandiose, but I mean it. Ok, I’ll restate it a little more humbly and a little more practically: We need to contribute to the future of techniques to analyze Big Data.
When all of us are sitting in rocking chairs in the 2030’s, we need to look back at the 2010’s—the decade (I’m convinced) will go down as the decade in which we started using information in profoundly new and clever ways to make better decisions—and know that part of the progress came from Chiliad. Humble, hmm?
Let’s return to today. With this month’s launch of Chiliad Discovery/Alert 7.0, we deliver the first step in that journey. Yes, “gorgeous” shows through, but so much more.
You’ll often hear us say:
With search engines like Google or Lucene/SOLR, the first thing you do when you get results is read. We believe that is the last thing you should do.
Behind this conviction is a core belief: In a Big Data world, there will always be too much information for you and I as humans to digest.
A quick story about scale: When I was an undergrad, we heard the tale of the student who proclaimed he would read every book in the main library by graduation—a library second only to the Library of Congress in breadth. Although voracious, he only got to books starting with “E.” And you know what? There were “only” 10 million books in that library. Here’s the mind-bending part: That’s a rounding error for our largest installation today. And 10 million documents is literally .00001% of the collection our latest customer will have in three years. So if only .00001% of that collection is applicable to some interesting question—and you know this is easily possible—then like that obsessive student (who probably skimmed anyway), you’d have to read 10 million documents. And you don’t get four years. You’ll usually have four days or four weeks…and in matters of life and death, you might only have four hours.
It won’t work. It doesn’t scale. We need our tools to help us extract themes, concepts, relationships, and more.
We do that. This is the Chiliad mission. Chiliad can Reach virtually any amount of information in any format spread all over your organization…or the planet. And we will Find what matters to you in seconds. But Reach and Find are only the FIRST two steps in our Iterative Discovery Cycle.
Why? Because when you have a lot of information, what matters to you really might fill a library. But when Chiliad finds what matters from that ocean, we don’t throw up our hands and leave you to the sharks. In fact, the first thing we show you is: What’s in there? What people? What concepts? What relationships and key dates? What places show up most? And how could that knowledge help me refine…or iterate…on my discovery? Our job is to help you solve the problem, not just find a bunch of documents.
What if that obsessive college student works for you? You could simply say: “Read all this and tell me what jumps out at you.” But instead of giving that ambitious scholar four years, Chiliad does this on the fly in four seconds—all
the important stuff relevant to your question formatted for quick review.
And yes, it looks gorgeous. But we believe this is a special brilliant, form of gorgeous. This user interface is the first step in a long journey to contribute to how humans deal with Big Data. But it’s a big step. We hope you will like it. On behalf of the Chiliad team, please let me know.
Ken Rosen is Chiliad Vice-President of Marketing.
Re “In a Big Data world, there will always be too much information for you and I as humans to digest.” I would suggest you might also want to consider curation in this context. The curation might not actually be done by Chiliad. For example, an analyst at the defense establishment might be the curator. Having said that, what hooks will Chiliad provide to hep this analyst curate?
Best,
Israel