Skip to main content

Thoughts on Language AGI Web Summit debate

I came across this youtube video https://www.youtube.com/watch?v=PBdZi_JtV4c&t=1008s. Excerpts from Web summit discussion titled “Debunking the great AI Lie” by Noam Chomsky, Gary Marcus, and Jeremy Khan.

The topic itself is very catchy and as an evangelist for “AI for Enterprise”, I was intrigued and listened to the 32-minute clip. At the outset, the arguments about language complexities, human and language connections, and how pure data-driven tools like GPT3, etc. are not sufficient to emulate full language comprehension were compelling. Gary mentioned the lack of sufficient early inputs from language experts in the development of the AI model. Chomsky used a very strong statement (even though he said he is being nice to AGI) about GPT3 and other types of language model research. He emphatically said these research efforts neither advance engineering capabilities (like telescopes in astronomy) nor advance scientific research on languages. However, he also mentioned that due to his hearing problem, he was able to understand questions from Jeremy using the real-time voice-to-text transaction. Gary also felt that $billions of research on autonomous AI were also a waste of money and effort. Gary also admitted that while these data-based models get it right 75% of the time, they can never get it right the remaining 25%.

I do not agree with the statement that these research areas and the products are a total waste. Research on autonomous cars is being used every day when you drive any mid-size car around the world. Parking assist, reverse assist, and lane change alerts are standard features in most cars and are used by millions of people. Language translators are helping millions of tourists around the world to do basic communication in foreign destinations. Voice-to-text tools like Alexa, and Siri have become standard appliances in most households.

We will leave the philosophical questions raised by the gurus about language nuances, syntax-semantic connections, the importance of past memories, etc for later and get back to our theme at @AiThoughts.Org i.e. “AI for Enterprise”. I have already argued in my earlier blog about how Artificial Narrow Intelligence is sufficient for the next level of enterprise automation and performance enhancement. I can reiterate by saying “Narrow AI is Deep enough for Enterprises” and we welcome all research efforts towards AGI which will eventually help in making ANI models much more accurate and solve enterprise problems.

Just to recall from my earlier Blog, I talked about HR-Operations certified expert NAI model able to understand most of the employee’s queries on HR-Operations issues and invoke the right ERP applications to complete the transactions. Gary’s assessment of 75% AGI correctness may translate to maybe over 90% in a limited domain narrow intelligence world such as HR-Ops, Invoicing Ops, Billing Ops, etc. I feel the research should not worry about reducing the gap of 10% but create an egoless model which can quickly realize that the employee’s query is not understandable by itself and transfer the call to a human agent in 10% of cases. How quickly and how seamlessly the model does the switch is what makes for a superior customer experience.

Hip-Hip-Hooray for the AI researchers both in AGI and ANI. Each will feed into the other to make AI models more accurate and usable. Enterprises have a huge amount of open source and commercial technologies and models available in ANI to get the best performance and they should allocate sufficient budgets and accelerate AIML projects in 2023.

More later,

L Ravichandran

 

 

Artificial Narrow Intelligence ( ANI) : Enterprises are OK with Narrow

Many of you have seen a news item from MIT https://news.mit.edu/2022/machine-learning-university-math-0803. The headlines was “New Algorithm aces University level Maths questions”. Lots of talk on Algorithms passing the undergraduate course in Computer science but due to their non-person status degrees cannot be legally given by Universities etc. Technically, these Algorithms, even though look so intelligent are classified as ANI as they are experts in only one area i.e. Maths or Computer Science.

Our mission and thoughts in @AiToughts.Org is always on #AIforEnterprise and We feel that Narrow is good. @sivaguru already talked about power of mixing AIML algorithms with virtual assistants in achieving much higher levels of automation. Let me illustrate this with a HR example. Many organizations have attempted to build HR Ops chatbots to improve employee experience. However, all these efforts have not achieved the full desired goals and full scale roll out and cutting HR Ops cost is yet a reality.

Let us use the MIT example and build a Algorithm who can pass HR Ops field. I have looked at various HR Ops orientation courses in large enterprises and also HR Ops courses in various MBA courses. Let the algorithm learn about all the technical nuances of HR policies, implementation, work flows and various nuances such as mixing of long weekends with vacation leave, health insurance issues, induction and exit process and bottlenecks. On top of the algorithm which is HROPs-Certified, sits a chatbot with local speaking nuances in the deployed country and a customized personal assistant.

Let us do a simulated run of this new HPOPs Certified ANI system ; let us call it HRANI

Scenario 1

Time 630am Local time : Employee shift is 900am to 500PM Local time.

Employee Peter: HI HRANI , I am sick

HRANI : Looks at the time and deduces that employee is at home and will not be able to come to work. Personal assistant logs in to HR ERP system and finds out employee has sick leave available. Personal assistant also checks the calendar and sees many meetings where the employee is accepted.

Hi Peter, Sorry to hear that you are sick. I am sure you will get well soon. You have sick leave entitlement and I have applied for sick leave. I have also sent a e-mail to your Boss xxx and your reportees mentioning about your leave of absence today. BTW, you have many meetings scheduled and do you plan to take them from home or shall I cancel.

Peter : Hi HRANI, Please cancel all meetings except 1200PM one.

Hirani : Done, Peter. Take rest. Hope you get better soon.

Scenario 2

Time is 100PM local time.

Peter : HI HRANI, I am sick

HRANI : Hi Peter, sorry to hear about your sickness. We have a campus doctor to consult and also a sick bay for you to rest. Shall I arrange this for you?

Same 3 short words from Employee “I am sick” is handled by HRANI in two different ways like a HR Ops person will do!.

Scenario 3

Peter : HI HRANI, I want to take off with family next month for couple of weeks

HRANI : Great Idea Peter. You need a break. You have not taken a single day off for last 9 months. There is a 3 days long weekend coming up and you can take vacation around it from dd/mm/yy to dd/mm/yy. You are short of 1 day leave and I can write a special approval request to your manager and with your track record I am confident that this will be approved.

Peter: Please go ahead.

HRANI knows about leave nuances of combining with weekends also knows about work flow for exceptions and approval probabilities.

Scenario 4

HRANI : Hi Peter, Happy to hear about your promotion and salary raise. I can suggest few ways to do better tax planning. Would you be interested?

Peter : Yes.

HRANI : Your PF ( 401K) contribution is Rs. Xxxx and if you take it up to Rs. Yyyy and if you donate RS. XXX to some charities, then your tax bracket will come down a notch and you save bundle of money.

Peter: Thanks, HRANI. I will consider this suggestion.

HRANI knows abut salary components, local tax laws and exceptions and can do a what if analysis and proactively advise employees similar to a HR Ops person.

This is Narrow as this only knows about HR Ops area. I am Good with it. We can also have ANIs for Billing Ops, Account receivables Ops etc. which take up bulk of operational costs in enterprises.

Can these scenarios possible in an HRANI ? The current NLP technologies to understand natural written text is pretty good and it is not that difficult to build algorithms to understand the Intent and also the context from free format human text. The various HROps scenarios are all written down in new HR employee orientation books and can be easily taught to algorithms by supervised learning methods.

To conclude, ANIs can make a huge impact to enterprises. I am OK with Narrow!

About Digital Twins – partly created by a digital twin!

One of the techniques popular among AI adoption and deployment initiatives is the use of Digital Twins.

While in its simplest form, they may be equated to simulations, there could be different levels of sophistication and applications.

In AI projects, particularly, Digital Twins can help accelerate the time to solution.

For example, in the real world, if one is modeling a prediction algorithm for failure recovery or preventive maintenance, gather actual performance data from physical devices may not only be time consuming, as one has to wait for a long time to reach some boundary conditions or gather sufficient data.

Using a digital twin, one may shoten that duration.

This approach would also be useful when, let us say, the device [an IoT end point?] is not easily accessible.

An extreme extension of this is when we can have digital assistants that can take some work load from us.

We would still want this assistant to be as close to our style and preferences in terms of how do things.

When I came across tools that generate articles, blog posts etc, wanted to try them out..

The article below has been generated by AI.

I used a tool called writesonic and chose to create this article in a guided manner.

I only had to type digital twin as the starting phase for the topic.

I answered a few questions to pick from 3-4 options suggested for the structure, style etc.. and bingo! This was generated!

 

 

Digital Twins: What They Are and Why You Need One

Image Source: FreeImages

Digital twins are digital representations of physical objects. They are also known as digital avatars, virtual twins, or digital companion models. Digital twins are becoming increasingly important in various industries because of the rise of machine learning and artificial intelligence (AI). This article will explain what a digital twin is, why you need one, and its many different types.

 

What is a Digital Twin?

A digital twin is a virtual replica of a real-world asset that, as the name suggests, is held in digital form. Digital twins are used to monitor and control real-world assets remotely and, depending on the type of digital twin, perform autonomous actions. A digital twin can be thought of as a virtual representation of a physical asset. For example, a car’s digital twin would include information about the car’s characteristics, conditions, and performance. A digital twin of a building may contain information about the building’s structural integrity and energy consumption. A digital twin is a computer simulation of a real-life object or process.

 

Why You Need a Digital Twin

A digital twin can be used to model a process or an asset to accurately predict future behaviour or performance. This can improve decision-making and planning by providing information about the asset’s or process’ current state. Digital twins can help you improve productivity, maintenance, and safety. They can improve collaboration by providing access to relevant data from anywhere, anytime. Digital twins can also increase trust by providing greater transparency and visibility into the asset. They can also reduce costs by enabling economies of scale and reducing the need for manual verification and auditing. Digital twins are key to the future of AI and machine learning. Digital twins can be used to train AI systems and to create new products and services. Digital twins are one of the most effective ways to bring AI to the enterprise. With a digital twin, you can create a tabula rasa environment free of bias, where AI can learn and grow without being influenced by human biases.

 

Types of Digital Twins

Physical Digital Twin – This is a digital representation of a product’s physical characteristics and design. It can provide information about a product’s design, origin, and transformation process. A physical digital twin is useful for supply chain management and quality control. Physical Digital Twin can be used to create Physical Digital Transformation (PDT) – a process of digitizing the supply chain, where raw materials are turned into finished products. Physical Digital Twin can also be used for parts management and maintenance. Business Digital Twin – This type of digital twin is business-oriented and includes the company’s structure and processes. It can be used to simulate business processes and see how they affect customers, suppliers, and other business partners. Business Digital Twin can be used to create a Business Digital Transformation (BDT) – a process of digitizing business processes and making them collaborative and autonomous. Business Digital Twin can also be used to create Business Process Automation (BPA). People Digital Twin – Features and information about employees. It can be used to simulate working and learning processes, identify potential problems in daily activities, and create customized coaching solutions. Person Digital Twin can be used to create People Digital Transformation (PDT) – a process of digitizing employee workflows, work instructions, and training materials.

 

Digital Twin Benefits

A digital twin offers many benefits to companies and customers. It creates better decision-making and planning by providing information about the asset’s or process’ current state. A digital twin can help improve productivity, maintenance, and safety. It can also help increase collaboration by providing access to relevant data from anywhere, anytime. Additionally, a digital twin can help increase trust by providing greater transparency and visibility into the asset or process. Finally, a digital twin can help reduce costs by enabling economies of scale and reducing the need for manual verification and auditing. A digital twin is key to the future of AI and machine learning. It can be used to train AI systems and to create new products and services.

 

How to Build a Digital Twin?

There are three main steps to building a digital twin: Define the asset that will be digitized – determine what assets you want to digitize. Build a virtual representation of the asset – define the assets’ purpose, functionality, and assets’ interrelationships. Connect the asset to other systems – link the asset’s virtual representation to other systems.

 

Conclusion

A digital twin is a virtual representation of a real-world asset. A digital twin can be thought of as a computer simulation of a real-life object or process. It offers many benefits to companies and customers and is key to the future of AI and machine learning. Digital twins are becoming increasingly important as AI and machine learning become more widespread.

Book Review: Radically Human

I recently read this book – Radically Human, by Paul R Daugherty and H James Wilson.

The authors are with Accenture and had written a book : Human + Machine: Reimagining Work in the age of AI, where they had discussed various case studies of how many organizations have been using AI, in the area of Human Machine collaboration.

In this book the authors propose five approaches – that they call as IDEAS as the acronym, that can be used to significantly improve the success rates of AI initiatives taken by organizations.

The key message is reassuring that AI is not to replace humans.

They talk about the first stage when the applications of AI was primarily to make routine and repetitive tasks automated. This led to discussions on whether AI will replace humans from their jobs.

The second stage was when AI approaches were used to augment or supplement human capabilities. This meant that humans could be much more accurate and much more productive. This calls for a more collaborative approach between humans and AI based solutions.

In order to achieve the benefits of this approach, one needs to relook at the five elements of innovation and by using many examples from their consulting experience with organizations across the world, the authors clearly demonstrate the adoption of these revised – possibly inverted – approaches to these five aspects.

Taken together, the IDEAS framework can be used for driving innovation across organizations.

IDEAs stands for: Intelligence, Data, Expertise, Architecture and Strategy.

While one should read the book to get a deeper dive into these factors, I will just summarize these for a quick reference:

Intelligence:

To make AI more meaningful and humanlike, one must think beyond the traditional deep learning approaches. Machines need to think and respond like humans. For example, Emotional AI is now finding applications in areas such as helping children in the autistic spectrum or in the autonomous automobiles. Two aspects they highlight are awareness and adaptability.

Data:

In order to address the struggle between the data scientists and the developers to have access to adequate data, for training or testing, approaches such as data echoing [to reuse data], active learning or synthetic data would be very useful

Expertise:

The shift that the authors propose is from ‘machine learning’ to ‘machine teaching’, where the humans can become more active teachers to help the machines learn. This will help understand a lot more context and the machine to become more intelligent!

Architecture:

The adoption of AI approaches widely has also led to the use of IoT types of devices that can constantly sense and generate a large volume of signals. For a system to be able to continuously learn and adapt to newer and emerging situations, it needs to be adaptive and scalable. This has an impact on the overall architecture of the systems. Adopting cloud based [if not cloud native] architectures and a distributed model of the systems [including microservices] to develop larger more complex solutions based on smaller components will help in making the architecture ready for the emerging future.

Strategy:

Ai technologies can significantly improve the friction usually found between strategy exercises and actual execution. With shorter cycles and closed feedback loops, strategies may be refined and fine tuned based on the execution experience. The authors call out three specific approaches that are effective for strategy. These are Forever Beta, Minimum Viable Idea and Co-lab approaches.

The book is divided into two parts.

The first discusses the IDEAS elements in detail and many examples under the theme of transforming innovation.

The second part is about competing in the radically human future, under four sub themes of Talent, Trust, Experiences and Sustainability.

The style of narration makes it a very readable book and I could learn a lot about the work already being done or that has been going on in the recent times, across the globe.

Overall, a highly recommended book for anyone wanting to get a quick update on the state of practice on AI adoption in enterprises.

Trends and challenges to consider for Enterprise AI solutions

Every company now wants to be an AI company or embed AI into their offerings.

While the maximum potential of AI has not yet been tapped, there are many innovative as well as controversial applications of technology that have been tried and some also in production.

We are already seeing simple applications of AI to make things easier for the humans using these solutions – from predictive text while composing messages or even documents (this post has been completely hand crafted and not generated by a machine 😊), or chatbots that do not get confused when the response does not fit one of the options proposed, to more complex applications such as self learning robots in rescue and assistive situations.

Many teams have been experimenting with revised ways of working to structure the software and solutions development processes.

Some of the challenges and proposed approaches are highlighted in these posts by Anil Sane on the aithoughts.org site.

https://aithoughts.org/aiml-solutions-security-considerations/

https://aithoughts.org/ai-and-aiops-a-perspective-for-it-services-industry/ 

The emergence of data science as a specialization has also meant that the software development lifecycle needs to acknowledge this and incorporate it in the flow.

Governance aspects of AI based solution development are also undergoing changes.

Some inherent issues related to AI based solutions – such as ethics, has been explained in the post by L Ravi in https://aithoughts.org/ai-ethics-self-governance/, also need to be considered as part of the overall approaches and be reviewed

The unpredictability of real time usage situations adds to some of the complexity of the algorithms to be implemented. It is no longer just an algorithm that processes data. The data determines or influences the behavior of an algorithm or even choosing the appropriate algorithm to be chosen at runtime.

Additional ‘-ities’ are also emerging for AI based solutions such as ‘explainability’ as non-functional requirements that need to be considered.

There are some topics related to the overall solutions supply chain that are still being explored.

Some of these, such as the bias in the learning data, aim to improve the overall quality of the decisions derived or suggested by AI systems.

As with any emerging technology, there are some bad actors who keep looking out for holes in the solutions to be exploited, that affect the social good.

Deep Fakes are a good example of advanced algorithms being used to mislead or even trigger anxiety or unrest in individuals or communities.

With the increased interest in the metaverse and related business, information is going to get even more distributed and fragmented.

This would then mean that any solution designer needs to think of ecosystems and not just point solutions.

We have already seen the advantage of this approach – such as using location services or map services offered by some companies being embedded in solutions delivered by many businesses.

Thinking of the ecosystem, one must consider the full supply chain and the potential for benefit and fraud at every stage.

These include challenges related to the bias in the learning data, data poisoning, deep fakes and compromises to the edge devices or sensors.

A recent report by CB Insights identified the following as emerging areas of concern and action, when it comes to AI. Here are the highlights, with my own thoughts also included.

  • Synthetic data for privacy.  This could also be very useful to increase the confidence in testing, where there is a constant struggle to share the real data between the developers and data scientists. It is not as simple to generate synthetic data and there are many considerations that go with it – to ensure adequacy, fairness [no bias] as well as constant validation of the neutrality of the data. We are used to capturing data mostly on the positive results. For example, We need to understand the patterns related to rejected parts during the manufacturing process, and that is another potential application of synthetic data – to generate images from quantitative or descriptive data that might have been captured in various analysis reports

  • Edge AI: embedding AI into the chips that would be the sensors [IoT devices?]. In addition to these being secure and immune to noise, some smartness would also be needed to ensure that these entry points are trusted

  • Policing or protection in the metaverse. While one may desire to have self regulation in the metaverse, one cannot wish away the need to have some mechanism of policing – essentially to create deterrents for abuse. Explainable AI and other such principles are useful in a reactive situation, but what is more important is to have some proactive mechanisms

  • Eliminating Deepfakes. We already spoke about this earlier in this article. When deepfakes are machine generated, the speed and volume could be a major challenge to combat

  • The report also talks about augmented coding. We are seeing tools from multiple vendors that are embedding intelligence in the development environment itself. For teams and organizations to learn faster, there would be a need to [selectively] share the learning across users. The question on how to tell the machine what may be shared and what may not be is another area that needs to mature

  • The next level of evolution of conversational AI, is to be able to support omni-channel interactions and multi modal AI, that can understand and process concepts from video, audio, text, images etc. this may be considered as the next evolution of natural interfaces and interactions, beyond just the spoken or written language

  • Black box AI – or end to end machine learning platforms would become the preferred option for enterprises to accelerate the adoption of company wide solutions

As seen from the above, the AI based solutions space offers enterprises unprecedented opportunities as well as unforeseen or unforeseeable complexities and challenges as the ecosystem also evolves.

In future articles, I intend to go deeper into some of the above trends and associated techniques or tools. The focus for me is to not lose human centricity when we embed more and more intelligence into the machines.

 

If you have any specific topics that you would like covered or questions to be answered, do reach out.

AI and AIOps – a perspective for IT Services Industry

AI and AIOps – a perspective for IT Services Industry

This write-up is to discuss about AIOps from the perspective of IT Services Industry and how possibly one need to shift towards bringing the benefits/efficiencies of AIOps at the Service Delivery / Service Consumption level.

There are definitive steps taken currently in the AIOps community to shift left from ITOPs to DevOps.

One can see that AIOps practices are making their headway into even pre-production activities with definite focus on predictive remedies, in order to build and deploy robust services.

This blog is looking at how AIOps is helping in the area of ITSM/DevOps areas and also brainstorming on how one could start integrating these practices / solutions into Service Delivery / Service Consumption areas.

Ever since Gartner coined the term AIOps, way back in 2016-17, the market has grown significantly and is expected to grow very fast in the coming years (one estimates that it will grow in the range of 20%-25% CAGR and may cross $40bn in next few years).

This phenomenal growth is attributed to

        Larger Digital transformation across IT Estates

        Varied and disparate platform / sources where the estates reside including cloud agnostic solutions

        Ever growing data across the estate (Engagement, Observational data)

        Larger, faster releases and deployments

The overall goal is to capture all data generated across the IT Estate, store, analyse, provide insights, and provide fixes thru appropriate automation. In this two aspects that play critical role are Big Data and Analytics thru Machine learning.

The following diagram is representative of how AIOps is playing role at various levels.

AIOps – In Operations

·        AIOps solutions are very strong in the area of ITOM, ITSM

·        Typical Solutions that are available currently

o   Domain-centric (domains like Application Monitoring, Log Monitoring, Network monitoring) (Examples of some products/Product companies Dynatrace, Datadog, ScienceLogic, S1 Platform, Zenoss, IPsoft etc.)

o   Domain-agnostic solutions available (works across disparate services and working across domains in IT environment) (Examples of some products/Product companies Big Panda, ServiceNow, BMC,  Elasticsearch, IBM Cloud Pak, CISCO App Dynamics,  Moogsoft, DataDog, Zenoss, Splunk etc.)

·    Personas: These are largely IT Operations personas, Service Delivery Personas – such as System Engineer, Site Reliability Engineers, Operations Engineer, Security Professionals, and Service Desk etc.

·    Process involves – Predict Service Failures, determine appropriate root causes and propose remediation and in some cases fix the issues before they affect the services.

·  Typical features involve Predictive Analytics, Predictive maintenance, Solution Recommendations, Creating knowledge articles, Intelligent Autoresponders, Persona based Analytics etc.

·    Some benefits are :  Proactively identify potential issues before they occur, remove noise from actual alerts that need attention, Improved IT Productivity, Improved  Utilization, Better visibility across IT estate, Optimize the spend across the estate, Better CSAT, Better relation with the business (from cost center to partner)

AIOps – in DevOps

·        As stated earlier, there is a trend to shift left – bringing AIOps practices / solutions to pre-production activities while Applications and Services Solutions are built, tested and deployed. This shift was imminent, given that Dev works very closely with OPS and has large impact in what gets designed, developed and deployed.

·        Persona – Developers, DevOps engineers, SRE Engineers

·    Some features includes – data ingestion for gaining insights while code is getting  developed and tested, proactively identifying anomalies in CI/CD pipelines, auto-remediation for such workflows (such as deployment of Pods and containers in multi cloud environment) using SLA for faster deployments and so on

·        Examples of Product/Product companies – Harness, Dynatrace, OverOps etc

·      Some Benefits are – better control on CI/CD stack, efficient use of pre-production estates, better resilient software solutions, robust design , Better productivity of DevOps community, and finally better integration with Ops

AI in Service Delivery/Consumption

·      Question is how to bring AI (and AIOps practice & discipline) into Service Delivery/Service Consumption areas and integrate the practices across Service Applications and underlying IT infrastructure to provide a complete integrated experience to the end user company / LOB owner

·        Few examples of  Service Delivery/Service Consumption could be

o   Selecting & Onboarding new resources onto Organization Platform

o   Talent Hunt with appropriate competencies

o   Allocation of competent human resource to a program

o   Allocation and management of Workplace / facilities to increase occupancy

o   Developing Market Strategy based on market / customer interests and Sales inputs

o   How to build a predictive Customer Service

o   Design and apply predictive maintenance needs in manufacturing setup

o   Detection of suspicious behaviour and persistent vulnerabilities that result in security threats across the ecosystems (& this is not restricted to IT Systems security threats but extended to threats to IPR, Knowledge Assets etc.)

·        Currently individual solutions do exist in the form of AI/ML solutions or robotic processes (including bots) in many areas including Customer Service, Healthcare, Finance, Stocks, Auto Industry, even fitness applications areas

·        In many case these, however, are not  integrated solutions or products within a given service delivery platform

·        While some of them can be integrated including digital workplace solutions (for example ServiceNow IT Service Management and ServiceNow HR Service Delivery), it will be imminent to bring the power, predictability and resilience of AIOps into Service Delivery functions too. This tight integration and convergence will help provide flawless, efficient Services to the end users.

·        It is evident that business has to spend time, money in meticulously planning for such tight end to end integrations in order to yield maximum benefit of automation at both the ends

·        Also, probably it is time now, to bring a certain standardisation in the mechanisms of doing such integrations

·        Some Examples of Product Vendors/Products :

o   Integrated HR Service Management : ServiceNow HR Service Delivery, DoveTail Employee Engagement  Suite, Oracle HR Help Cloud Service, SAP SuccessFactors Employee Central Service Center,

o   Workspace Management : ServiceNow Workplace Service Delivery,

 —————————————————————————————————————————————————————-

References:

·        Gartner Market Guide for AIOps Platforms

·        ServiceNow Workplace Service Delivery, HR Service Delivery

·        Mordor Intelligence – Market Snapshot