Cutting Through the Hype — Essential Questions to Ask Before Hiring an A.I. Company
Let’s face it: in our fragmented and complex industry, construction buyers often lack the sophistication needed to navigate new technologies.
Currently, A.I. technology is the biggest trend of all and it’s impossible to ignore its impact. Recent advancements in large language models are not hype; they’re real, and construction companies that fail to adapt will be left behind.
Put simply, being sophisticated about screening A.I. solutions should be a new skillset—one that every organization learns to do well.
In the sections that follow, I’ll outline four essential lines of inquiry that will put you at an advantage during discovery calls with A.I. companies. This diligence will:
- Conserve your resources during the screening process, preventing unnecessary follow up calls and exhaustion.
- Differentiate legitimate A.I. companies from the ones with only “A.I.” sprinkled on.
- Avoid the potentially very large opportunity cost of choosing an unsuitable provider.
The A.I. floodgates have opened
Everywhere you look, there seems to be a new product with “A.I.” slapped on it. It’s gotten to the point where the term is starting to lose its meaning, leaving many of us feeling overwhelmed.
The release of ChatGPT by OpenAI in November 2022 changed everything. Since then, we’ve witnessed an explosion of interest in A.I. from all corners of society.
Foundational models, such as GPT-4, Google Gemini, and Anthropic’s Claude are extraordinarily expensive to train, yet they deliver remarkably conversational responses on a wide range of topics and tasks.
These models are versatile and powerful, making them incredibly useful for various applications, including in industries like construction where true A.I. was previously unattainable. Today, foundational models are what most people think of when they hear “A.I.”
However, this moment is defined not just by the power of these foundational A.I. models—it’s the fact that they are so easy to use and integrate at scale.
Two types of A.I. companies
In the Placer Solutions industry research report on Text-Based Artificial Intelligence in Construction, we outline several levels of implementation for 45 different construction activities.
At its simplest, this starts with a simple hook into OpenAI’s API, a task that takes a developer just a few hours and essentially duplicates what can be done through web-based ChatGPT.
This ease of implementation has distorted the marketplace because software companies can finally, for the first time ever, legitimately slap “A.I.” on their marketing and be somewhat truthful about it.
Put another way, the integration of foundational A.I. models into software is table stakes now.
I think we need a new way of thinking about this. Looking ahead, there will be two types of software companies:
- Integrators: These companies incorporate foundational A.I. models like GPT-4 into their products, often adding just a superficial layer of A.I. to their existing solutions.
- Innovators: These companies fundamentally enhance or modify foundational A.I. models, offering deep, unique technological advancements.
There’s a spectrum here—and this is the grey area that marketers love!
In this new paradigm, many companies that only perform simple integrations will label themselves as “A.I. companies.” This not only cheapens the concept of A.I. but also risks degrading the user experience with subpar performance.
The risks of inefficient and hasty adoptions will be unclear, and you’ll still bear the cost of hallucinations and data concerns that come with large foundational models.
What not to do
More likely than not, this vendor is not differentiated from others on the market. In many cases, what the seller offers at a premium, the buyer could accomplish on their own.
The vendor might claim that they operate A.I. on the builder’s data, which sounds impressive, but this is extremely trivial right now. Meaningful A.I. requires more construction-specific workflows and deep expertise that is competently implemented by A.I. experts.
The problem is that the buyer failed to ask pointed, specific questions that would have very quickly discerned some key insights about the product.
So what questions should you be asking?
“What can your A.I. product do that ChatGPT can’t do?”
This is the simplest yet most effective question to ask. You may need to ask it several times, especially if the provider is being dodgy about answering.
Look, the simple truth is that unless you’re Google or OpenAI or Meta, you’re probably not building a foundational model — and none of those Big Tech companies are building anything specific for construction.
So we’re stuck with the fact that nearly every construction A.I. product on the market today is going to be integrating one of those foundational models.
In many cases, and this is where you’ll need to be savvy, their A.I. will simply be a thin layer on top of something like ChatGPT. The term ‘thin layer’ means that the software adds minimal unique functionality or enhancement to the underlying A.I. technology.
Essentially, these companies are using a powerful existing model like ChatGPT, wrapping it in a minimal interface, and branding it as a new product. This superficial layer might handle basic tasks like interfacing with the user or tweaking the A.I.’s responses, but it does not significantly alter or improve the core A.I. capabilities.
There’s nothing bad about a product that integrates ChatGPT, but there’s nothing unique about it either — and calling yourself an “A.I. company” based on that is disingenuous.
Follow-ups for this line of questioning:
- “What specific enhancements or modifications have you made to the base A.I. model you’re using?”
- “Can you provide case studies or examples where your product outperformed standard A.I. models like ChatGPT in similar applications?”
- “Are you able to elaborate on any technology or algorithms that you’ve developed that are not found in common A.I. models?”
Why it’s important:
- You shouldn’t be paying a premium for a software product that doesn’t accomplish anything more than what you can get by opening a Microsoft Azure account and creating your own secure instance of GPT-4.
“What is the A.I. expertise of executives at your company?”
You can ask this one directly or do an online search to figure it out. Either way, if a company claims to be an “A.I. company” then they need to have A.I. expertise in their organization’s leadership. Period.
Firstly, don’t be immediately impressed by name brands. If someone says they “did A.I. at Google”, dig into what this means. Those big companies basically have A.I. everywhere in their org.
For example, it might be that the founder worked on an internal billing product and was integrating A.I. from another team to detect fraud. That person can reasonably claim that they “did A.I.”, but they really have no machine learning bona fides whatsoever.
Secondly, there’s plenty of reasons that the CEO of a tech company doesn’t need to be an A.I. expert. There’s a lot of software that’s useful beyond A.I. applications!
But A.I. is different. If a company claims to be making an actual A.I. play then they need to have some very deep expertise at the CTO or CEO level.
The backgrounds you’re looking for include: PhD in machine learning or a related field, significant experience in machine learning and neural networks, a history of published research in reputable A.I. journals, prior involvement in successful A.I. initiatives.
This level of expertise ensures that the leadership is not just familiar with the technical aspects of A.I., but is capable of understanding the complex challenges associated with deploying A.I. in construction, and building a product that fundamentally learns your construction workflows.
Follow-ups for this line of questioning:
- “Have any of your founders been involved in notable machine learning projects in the past?”
- “Can you describe a challenging A.I. problem that one of your founders has successfully solved?”
- “Are there any technology collaborations or partnerships that your leadership has forged based on your machine learning expertise?”
Why it’s important:
- You want to be working with A.I. companies that understand the complexities of A.I. and who are committed to delivering robust, effective solutions, not ones that overpromise and underdeliver.
“How focused is your A.I. on construction problems exclusively?”
This questions riffs on the the first two. After you get it out of the way that the provider is leveraging existing foundational models, the question then becomes how they’re applying their A.I. expertise to construction problems.
You see, the corollary to the earlier point about Big Tech being the only ones making foundational models is that those Big Tech models are themselves very vanilla. They’re trained on petabytes of Internet-scale data that makes them liable to hallucinations and generic responses. They lack any context whatsoever about what a construction project is.
This means that Microsoft Copilot and Google Gemini fail here.
So the provider’s play, assuming they are an actual A.I. company, is to improve these foundational models in some way that addresses one or more underlying challenges in construction.
Follow-ups for this line of questioning:
- “What feedback mechanisms do you have in place to ensure your A.I. continually learns from its deployment in my construction environment?”
- “How do you measure the impact and effectiveness of your A.I. in construction settings?”
- “What are the limitations of your A.I. technology in addressing construction-specific issues, and how are you working to overcome them?”
Why it’s important:
- Before entering into a partnership with an A.I. company, it’s important to uncover how deeply and effectively the provider has tailored their technology to meet the demands of the construction industry, beyond just basic application of a generic A.I. model.
“How is my data handled and treated?”
This question gets to the heart of a big challenge in machine learning: the need to leverage large amounts of data, oftentimes proprietary, that might be comingled with other people’s data in uncontrolled ways.
A point worth making here: I’ve heard of construction companies who refuse to work with technology companies and instead embark on large-scale internal data initiatives to “protect their data”, only to be disappointed in three years’ time when they’ve spent millions of dollars on consultants and failed to gain business lift.
So this isn’t intended to make you paranoid and refuse to hand over any of your project data to tech companies. You just need to be smart about it.
You want to work with providers who understand that your data is your data, not theirs; companies that have controls in place to silo your data from others; and you want to be clear on how they’re improving their models using your data, specifying which data can be used for training and which cannot.
On your side, you need to understand your own data needs, which may or may not vary on a project-by-project basis depending on your own owner requirements.
Follow-ups for this line of questioning:
- “Can you detail whether my data will be segregated from other clients’ data?”
- “Who within your organization will have access to my data, and under what circumstances?”
- “What rights do I retain over my data once it’s in your system?”
Why it’s important:
- You need to ensure that the provider not only respects the privacy and ownership of your data but also adheres to best practices in data management and compliance.
We operate in an industry where communication and collaboration is key— modern large language models are very good at this type of thing.
As the technology advances, adopting A.I. in construction is no longer a matter of if, but when.
Companies that move quickly to integrate A.I. will gain a significant advantage, while those who delay risk falling behind. It’s essential to act now to secure a strong position in a market that’s rapidly changing with A.I.’s influence.
In the process, you need to be smart about screening A.I. companies and not get distracted by shiny objects amidst all of the marketing hype.
Nate Fuller is a passionate and accomplished construction technology leader with a diverse background in corporate innovation, construction technology, and entrepreneurship.
His proven track record defining strategy and directing change management in construction has helped North America’s largest construction contractors build and scale effective technology programs.
If you found this article insightful, please consider signing up for my newsletter at placersolutions.io
