Palantir’s CEO, Alex Karp, recently visited Japan in order to discuss Palantir & the future relationship with the nation.
Investors within Palantir may know that the innovative data mesh product for Palantir has been used within Japan firstly, before later deployment around the globe.
According to reports, the reason for the visit was to discuss mainly the data mesh capabilities: “Palantír CEO Alex Karp (centre of the photo) is visiting Japan. On this day, we paid a courtesy call on former Prime Minister Shinzo Abe and exchanged opinions.”
Palantir Japan will provide “Palantir Data Mesh” from 2022 this year. Data Mesh is based on Foundry and can integrate data from data warehouses (DWHs), data lakes with unstructured data, and various other legacy systems.
The biggest feature is that a pay-as-you-go price has been newly set. It supports data migration to Data Mesh for free, and there is no storage usage fee for the storage destination. Since there is no minimum usage period or additional charges, not only large companies but also small and medium-sized enterprises can start using the service with a small start with less risk.
James Neil of the company said, “Providing Data Mesh in advance in Japan is based on our desire to deal with more Japanese companies. Try to work in the Japanese market and proceed step by step. I found that many organizations think that it is necessary to continue. I believe that this Data Mesh will support the modernization of the data infrastructure of more Japanese companies and bring about change.”
Palantir Technologies has deep relationships with Japan. Back last year Palantir announced that they were awarded a $22.5 million, one-year contract in partnership with SOMPO Holdings for the “Real Data Platform for Security, Health, and Wellbeing”.
“SOMPO’s visionary Real Data Platform, or “RDP” is a collaborative ecosystem between public and private sectors that will improve healthcare in Japan, streamline supply chains across industries, and increase security and resilience in the region.”
Alex Karp, CEO at Palantir mentioned that he had been a follower, and an admirer of Japan & the nations culture for some time. In all areas, “movies, literature & martial arts”.
Alex Karp said, he is excited to enable the Japanese to benefit from the Palantir software solution.
What is unique about Japan?
Palantir Japan, which has a high track record in the world, was established in Japan in 2019 in partnership with SOMPO Holdings. The reason for creating a joint venture is that Japan has a wealth of real data.
According to media reports, OMPO and Palantir will work together while a business model that makes use of this has not yet been created, and there is a possibility that it will be possible to create solutions that were unthinkable in Japan and disseminate them to the world.
Mr. Katsuyuki Ohara of Parantia Japan Head of Growth said, “Although it is often said that Japanese DX is late, there is a huge amount of complicated data acquired through advanced design and engineering in Japanese industry. It is exactly Japan. Is a treasure trove of real data. With Foundry, you can consolidate these data in one place and make full use of it in your company’s decision making. “
Interestingly, according to recent data Japan is facing an aging population crisis, and issues associated with healthcare management.
The Japanese super-aged’ society is the oldest in the world. Over, 28.7 % of the population are 65
or older, with women forming the majority. The country is also home to a record 80 000
centenarians. By 2036, people aged 65 and over will represent a third of the population.
Palantir has been fundamental towards the British vaccination rollout, achieving the best rollout statistics to date.
Louis Mosley, head of Palantir UK, said: “We are proud to be able to support NHS England and the wider elective recovery plan. Palantir’s software helped to deliver the COVID-19 vaccine programme and is now assisting NHS staff in reducing the backlog and delivering the highest standards of patient care.
“Our software is trusted by some of the most important public and private sector institutions around the world, in part because of its advanced data protection tools
A case study of Chelsea and Westminster outlined in the plan, says Palantir’s software has enabled a 28% reduction in the inpatient waiting list through validation and better clinical oversight.
It adds that booking lead times have tripled from six to 17+ days on average, meaning patients can be notified about surgery in a timelier manner and reducing the number of cancellations due to lack of staff or patient availability. Also, theatre utilisation has improved from 73% to 86%, over a three-month period.
This shows the relevance Palantir can achieve within the health & data scene.
Within a recent article released by the Japanese Government, the Japan’s Growth Strategy includes ambitious plans to build on strengths in infrastructure and devices to tap into the market for big data services and become a world leader in IT.
Furthermore, a boost in national capability will be needed to overcome the shortage of data scientists.
As mentioned within the report, “privacy of individual personal data will be a key challenge.”
To add, the Japanese data growth strategy in June aimed to reinforce Japan’s position as “a world-leading IT nation.”
The government predicts the market for “Big Data” services will be £51 billion (7.7 trillion yen) by 2020.
“It has allocated £87.5 million (13.2 billion Yen) for Big Data R&D, including projects to develop a 400Gbps new hi-speed network infrastructure and Test-bed, a high-availability and high-efficiency data centre operating system, and data analysis applications.”
Renewed technology infrastructure should boost Japan’s industrial competitiveness and create new industries and innovation.
What does Japanese media think of Palantir?
Japanese media stated that, “what sets Palantir apart from other big data analytics companies is the incorporation of strong privacy management and governance capabilities into its solutions.”
Furthermore, the Japanese report said “the company’s solution was originally created after the 9/11 terrorist attacks, specializing in national security, and has been used by US defense agencies.”
Palantir, according to Japanese media has been active in finding patterns in large amounts of data, identifying information related to terrorism, and protecting soldiers from blasting devices.
Palantir Data Mesh:
The Palantir data mesh has been used within Japan for some time. Investors expect that Palantir will release this product more aggressively within the coming years.
Palantir explains: “Palantir Data Mesh can upgrade historical investments in data warehouses, data lakes, and other legacy infrastructure to unlock additional value.”
“By meeting the enterprise wherever they may be along the digital modernization journey, Palantir Data Mesh seamlessly integrates the enterprise’s legacy systems through its interoperable architecture and automated deployment.”
Alternative platforms charge customers to put their data in and get their data out — and the solutions themselves are insufficient. This includes the data warehouse, and data lakes:
1) Data warehouses rely on known schema, neat hierarchies, and rigid formats to store structured data, leaving little room for flexibility.
2) Data lakes are designed for basic query and analysis — but data legibility and usability is crippled by exponential data volume.
“Palantir Data Mesh comes with data connections to hundreds of source systems. It allows domains to combine this source data, along with models of the business, to form an operational digital twin of the enterprise. The Mesh automatically deploys and manages the infrastructure required to build and refresh these domain-produced data objects.”
Furthermore, the product automatically generates and stores pipeline code and lineage as an integrated part of each dataset — and automatically builds, deploys, and runs the code for data products. In addition, data objects contain instructions on how they are to be used and combined, enabling each domain to use data independently while all data in the Mesh remains wholly in control of the enterprise.
Palantir Data Mesh provides multiple planes of infrastructure to support any type of user — including software-driven data integration, templates for common data pipeline workflows, auto-scaling of computation nodes, and more.
The Mesh empowers data product developers to independently release and globally federate new data products. To keep this tightly organized and easily accessible, the Mesh also automatically forms a graph of globally connected data products, each still owned by individual domains.
Palantir Data Mesh automatically standardizes interoperability across domain objects, while also federating updates back to legacy / source systems.
To address the automated decision execution requirement of this principle, the Mesh also enables ML, AI, or other logic-based decision automation that not only captures the full context surrounding those decisions, but also enables global scenario evaluation and cross-domain optimization.
There Are 5 Main Principles To Understand The Data Mesh:
The 5 main principles of the data mesh is split into organisational, architectural, technological, operational, and principal changes, in comparison to the current conventional methodology.
- Organizationally, it shifts from centralized ownership of the data by specialists who run the data platform technologies, to a decentralized data ownership model pushing ownership and accountability of the data back to the business domains where it originates from or is used.
- Architecturally, it shifts from collecting data into monolithic warehouses and lakes to connecting data through a distributed mesh of data accessed through standardized protocols.
- Technologically, it shifts from technology solutions that treat data as a by-product of running pipeline code, to solutions that treat data and code that maintains it as one lively autonomous unit.
- Operationally, it shifts data governance from a top-down centralized operational model with human interventions, to a federated model with computational policies embedded in the nodes on the mesh.
- Principally, it shifts our value system from data as an asset to be collected, to data as a product to serve and delight the users.
Why Do We Need A Data Mesh?
From a macro-perspective, there are evident drivers in which have caused society to move towards this new initiative of a data mesh. Firstly, the evident increase within business complexity, combined with uncertainty, proliferation of data, and the overall variability of data from a range of different sources.
This is in addition to the necessity for business agility, the importance of getting value from data and the importance for organisations to change and become resilient.
Therefore, this means that society now has a vital decision to make. If as a society, we will continue with the current convectional approach, there is an evident danger of plateauing, and therefore failing to capitalise on the utility of data.
However, if we use the data mesh approach, this will mean that organisations will capitalise on the reality of the data trend, in turn being able to gain value from data at scale.
What Are The Issues With The Conventional Model?
1) Organisations today use a centralised strategy to process extensive amounts of data, with various sources, types and overall use cases. However, centralisation requires users to transport data from edge locations to a central data lake to be queried for analytics. This innately is time consuming and very expensive.
Interestingly, a data mesh can solve this issue because, the data mesh is a decentralised ownership model in which reduces the time-to-insights, and time-to-value by empowerment of teams to access and analyse non-core data quickly and easily.
2) In addition, as the growth rate of data expands exponentially, the current query methodology used within a centralised management model requires changes to scale.
There is an evident slowdown in the response time to new data sources, as the overall sources increase within the conventional model. This is negative for a business whom want to gain agility in order to derive the most value from data.
Once again, the data mesh can solve this issue because data mesh delegates ownership from centralised locations, to the individual teams or users in which enables agility.
Furthermore, this means that businesses can experience real-time decision making by closing the time and space between an event occurring, and the analysis.
3) Another issue within the current model is the susceptibility to data residency and the guidelines in which prohibit data migration if the data is stored within a certain geopolitical region, or if the data is stored within the EU, however needs to be accessed by a user within another nation.
Aiding by data governance regulations can be costly and time consuming, therefore delaying the overall time for businesses to analyse data.
Within the data mesh, there is a decentralised data management system in which domains are responsible for the quality, security and transfer of their data products.
Via this decentralised approach, there is a connectivity layer in which enables direct transfer access to data sets where they reside, in turn avoiding costly data transfer issues, and residency concerns.
