Palantir Chief Architect Akshay Krishnaswamy and Head of AI/ML Solutions, Anirvan Mukherjee explained within a recent exclusive interview the functionalities behind Palantir AI & ML.
Palantir focuses on a few key areas:
- Closing the operational connectivity gap.
- Foundry approach to AI/ML
- Three step framework for connecting AI/ML to operations in Foundry
In terms of closing the operational connectivity gap, the Chief Architect said that many organisations lack the overarching framework to ensure that AI/ML/OTP can be embedded into operations – resulting in fragmentation and limited learning.
Given this fragmentation, this is a major issue when doing things at scale.
Foundry enables full stack data science, said Akshay. Palantir can enable and accelerate time to production.
This includes integration of all data from a range of different sources, ability to build robust pipelines at scale. Overall this blends into an end-to-end model building and training feature-set, said the Palantir employee.
One can integrate models from partners, such as Microsoft & other partners too.
- This means that, the Foundry modelling framework eliminates notebook refactors, irreproducibility and orphan ML models.
- Furthermore, Palantir “keep models fresh”. This refers to live connectivity with production data pipelines in which can enable up to date models on the newest pieces of data.
- Operationalisation of models is key, said the executive. Users can deploy new models into applications directly with a few clicks. To add, share lineage and security frameworks apply across teams.
- And to finalise, Palantir enables feedback loops between consumers and model builders. The continuous capturing of operator feedback on deployed models can enable data science teams to iterate and improve faster.
We have spoken to individuals in a range of industries, such as Oil & Gas & AirSpace. “Capturing operational feedback is a very non trivial task”, he said.
To summarise, the Chief Executive mentioned how Palantir can integrate existing data assets and models using a suite of interoperable connectors to external systems.
Secondly, the company can bind models to digital twins (ontology) in which is therefore specific to your organisation. These models can be injected directly into core applications.
Thirdly, operationalisation of data occurs. One can operationalise their data to write back decisions to source systems.
Does Foundry have capabilities to build AI/ML models?
Anirvan said, yes – we have a range of technologies in which makes this easy. Palantir gives one the ability to build these models as part of the cohesive data mesh in which is being built out.
One powerful thing in Foundry is the ability to branch an entire model and data pipeline.
This means that, individuals using Foundry can simulate, change and predict how one change will impact a model. One can try this all out within Foundry.
Palantir tries to make sure that everything is supported, said the Head of AI and ML solutions.
The interconnection of AI & ML models is very powerful. These functionalities must be interconnected in order to solve problems.
How does Edge AI relate to Palantir?
Foundry is a very rich place where many workflows take place. But, certain models are best deployed at the edge. These are instances in which a system is automatically making decisions.
What is a satellite sensing, and should this mean the satellite takes autonomous action?
Palantir is the orchestration layer in which can make this possible. The Foundry model evaluation functionalities are key for edge environments. One can simulate how an edge device will perform in a certain situation.
Palantir has a rich set of tools to deploy AI models and other logic at the edge.
This is a very exciting piece, said Akshay. Edge devices can unlock a lot of value, he hinted. The full chain is often needed to push intelligence to these edge devices. Palantir can solve this.
Something exciting is to unlock use cases at the edge, via Palantir Foundry.
Society is moving into a new paradigm, namely the ideology of Industry 4.0. Industry 4.0 refers to the deployment of a large amount of sensors, and actuator devices forming an Industrial Internet Of Things.
The IoT speaks towards the extension of the application of internet communications beyond computers and networked devices, to also include the networking of everyday objects. In other words, a network of physical objects in the future will have embedded software, sensors and other technologies to connect and exchange data with other devices and systems over the internet.
In consideration of the huge amounts of inter-connectivity and data being collected by objects within the future, edge compute is required given the huge amounts of data produced by the IoT devices.
In accordance with an array of reports, the Edge AI Computing market is no trivial feat, and instead presents a $59.63B market size projection by 2030. This is mainly driven by the necessity to overcome cloud computing challenges based on the rise in demand for real time operations at the edge.
Within the context of manufacturing, businesses can embed Palantir Edge AI on sensors and cameras in manufacturing plants — as well as on machines in processing factories — to increase efficiency and improve quality control. When these sensors examine components as part of a production line, models deployed with Palantir Edge AI identify defective products and separate them for inspection or repair.
Interestingly, when looking at autonomous navigation, there is also a major benefit that Palantir can bring: “for autonomous vehicles, algorithms deployed on vehicle sensors with Palantir Edge AI detect potential obstacles in the path of the moving vehicle, and command the control system to avoid or mitigate collision.”
Overall, Palantir is becoming more than solely the Operating System for the new enterprise. Palantir instead, is becoming the base layer for Industry 4.0. If organisations need to transition towards this new paradigm within society, namely more interconnectivity due to the IoT & interoperability of clouds, Palantir has the Operating System ready to use.
How does Palantir aid active learning?
Palantir thinks, how can one identify and adapt changes in which are vital to solve for, and how can one be the most opportunistic to evolve and adapt models?
From a Foundry perspective, a data mesh empowers this.
Data feeds occur from real world life changes, and therefore can be understood in detail. Furthermore, one can overlay other data on top of this. One can understand what is going well, and what needs to be improved via this overlay.
Speak to the cyber security standards of Foundry?
Akshay said, “he is very excited to share this news”.
Whilst Palantir is not a sole cybersecurity product, through the operating system, Palantir has the ability to understand the connections and relationships between data points, and therefore can aid analysts to investigate the origins sand features of cyber attacks. This can enable a devised, highly tailored response by allowing organisations to diagnose attacks and take pre-emptive action against future cyber threats.
One main axiom of Palantir Cyber is via anomaly detection capabilities. This means that, analysts can begin cyber threat investigations in Palantir by combing through massive amounts of data to find anomalous occurrences. This data can be filtered through with sub-second querying of trillions of records at petabyte scale.
Furthermore, using the technology, companies can iterate and build on strategic algorithms to comb through data archives, and to detect anomalies by creating clusters that reveals previously unknown entities, events and connections.
context of organisational software solutions. Tao stated, that “security by design should become a premium product and people over time will learn it is in their interest to pay for it.” Palantir, innately have security features within their products since inception.
Palantir stated within a commentary that, “given the critical work performed on our platforms, information security is our lifeblood. Our industry-leading InfoSec team works tirelessly to stay ahead of adversaries by hunting for sophisticated threats, thwarting changes in their tactics, and immediately eradicating risks.”
Palantir stated that, “recognizing that commercial institutions face a shared set of cyber threats, we created the Cyber Mesh, a platform for secure information sharing among peers. Drawing on successful models within the defense and intelligence communities, the Cyber Mesh enables secure peer-to-peer sharing between enterprises with automatic redaction of sensitive data.” This indicates the overall collaborative environment in which Palantir poses forward. This specifically addresses Stock, and his concerns in regards to limited collaboration.