A Global Engineering Consultancy
Transforming Data Access for a Global Engineering Firm
Implemented Elasticsearch to reduce search response times from 10–15 seconds to under 500ms across 200+ PostgreSQL tables of technical BIM, CAD, and project data.
Background
The Problem
A leading global engineering consultancy specialising in BIM, Structural, and MEP solutions managed vast volumes of technical data: CAD and BIM design files, project documentation, structural specifications, client histories, and email communications — all spread across 200+ PostgreSQL database tables. Basic searches took 10–15 seconds, limiting productivity and collaboration across global offices.
Challenges
Data Complexity
200+ PostgreSQL tables with diverse data types — structured, unstructured, and binary — and complex relationships between data entities. Different user roles required different access control at the search result level.
Technical Constraints
Zero downtime during migration was a hard requirement. Data consistency had to be maintained throughout the transition, and new data needed to be indexed in real time as it was added.
Global Scale
High volume of historical data, high frequency of new additions, and performance requirements that had to hold across global offices in different time zones.
Our Approach
Elasticsearch Schema Design
Created custom Elasticsearch mappings for each data type — documents, emails, BIM files, project records — with permission-aware field configurations to enforce access control at the search layer.
Distributed Cluster Architecture
Implemented a distributed Elasticsearch cluster configured for the firm's data volume and query concurrency requirements, with monitoring via Elasticsearch Stack and Kibana dashboards.
Real-Time Indexing Pipeline
Developed incremental indexing pipelines to index new data in real time as it was added to PostgreSQL, with validation processes to ensure data integrity throughout.
Advanced Search Features
Implemented fuzzy matching for technical terms, faceted search for project attributes, real-time search suggestions, and permission-filtered results to surface the right information to each user.
Outcomes
Search Speed: 10-15s → <500ms
Basic searches that previously took 10–15 seconds now return results in under 500 milliseconds — a 20–30× improvement in response time.
Improved Productivity
Engineers and project managers gained faster access to technical data, improving project coordination and reducing time spent hunting for information.
Scalable Foundation
The Elasticsearch architecture scales with growing data volumes, providing a search infrastructure capable of supporting the firm's future growth.
Better Monitoring
Kibana dashboards and Elasticsearch Stack Monitoring gave the operations team full visibility into cluster health and query performance.
Conclusion
This engagement demonstrates how the right search infrastructure can transform knowledge-intensive businesses. For engineering firms managing complex technical data at scale, purpose-built search is not a nice-to-have — it is a productivity multiplier.
Have a similar challenge?
Tell us about your project. We will tell you honestly whether we can help and what a realistic outcome looks like.
Discuss Your Project →
