Skip to main content

Tag: Large language models

Domain and LLM

Posted on October 27, 2023 - 4:45 am by

I am in total agreement with Morgan Zimmerman, Dassault Systems quote in TOI today.  Every industry has its own terminologies, concepts, names, words i.e Industry Language. He says even a simple looking word like “Certification” have different meanings in Aerospace vs life sciences.  He recommends use of Industry specific language and your own company specific language for getting significant benefit out of LLMs. This will also reduce hallucinations and misunderstanding.

This is in line with @AiThoughts.Org thoughts on Domain and company specific information on top of general data used by all LLMs.  Like they say in Real Estate, the 3 most important things in any real estate buy decision is “Location, Location and Location”.  We need 3 things to make LLMs work for the enterprise.  “Domain, Domain and Domain”.   Many of us may recall a very successful Bill Clinton Presidential campaign slogan. “The economy, Stupid”.   We can say “The domain, Stupid” as the slogan to make LLMs useful for the enterprises.

But the million-dollar question is how much it is going to cost for the learning updates using your Domain and company data?  EY published a cost of $1.4 Billion which very few can afford.  We need much less expensive solutions for large scale implementation of LLMs.

Solicit your thoughts. #LLM #aiml #Aiethics #Aiforindustry

L Ravichandran

AI Legislation: Need Urgency

Posted on October 20, 2023 - 4:50 am by

Let me first wish you all a happy Navratri Festivities.  I still fondly remember the Durga Pooja days during my Indian Statistical Institute years.  However, we also need to remember we are in the midst of two wars, one in Ukraine and other in Middle East.  We wish solutions are found and further loss of life and destruction is stopped.

I came across two articles in Hindu Newspaper regarding our topic AI.  I have attached a scan of an editorial by M.K. Narayanan, well known National Security and Cyber expert. 

Few highlights are worth mentioning for all of us to ponder.

  1. There is a general agreement that latest advances in AI do pose a major threat and need to be regulated like nuclear power technologies.
  • All countries are not only “locking the gates after the horse has bolted””, but “discussing about locking the gates and deciding on the make & model of the Lock while the horse has bolted”.  Huge delays in enacting and implementing AI Legislation is flagged as a big issue.
  • Rogue nations who willfully decide not to enforce any regulations will get huge advantage over law abiding nations.
  • More than 50% of the large enterprises are sitting on “intangible” assets which are at huge risk of evaporating by non-state actors with AI powered cyber warfare.
  • Cognitive warfare using AI technologies, will destabilize governments, news media and alter the human cognition.
  • This is a new kind of war fare where state and technology companies must closely collaborate.
  • Another interesting mention of over dependence on AI and algorithms which may have caused the major intelligence failure in the latest middle east conflict.

All of these point to the same conclusion.  All countries and multi-lateral organizations such as UN, EU, African Union, G20 etc., multi-lateral military alliances like NATO etc. must move at lightning speed to understand and agree on measures to effectively control and use this great technology.  

The old classic advertisement slogan “JUST DO IT” must be the motto of all the organizations.

Similar efforts are needed by all large enterprises, large financial institutions, regulatory agencies to get ready for the scale implementation of these technologies.

Last but not the least, large technology companies need to look at this not just as a form of another innovation to help automation, but a human affecting , major disruption causing technology and spend sufficient resources in understanding and putting sufficient brakes to avoid run away type situations.

Cyber Security, Ethical auditors, risk management auditors will have huge opportunities and they have to start upskilling fast.

More later,

L Ravichandran.

AI Regulations : Need for urgency

Posted on August 18, 2023 - 5:57 am by

Few weeks ago, I saw a news article about risks of unregulated AI.  The news article quoted that in USA, Police came to a house of a 8 months pregnant African American lady and arrested her due to a facial recognition system identified her as the theft suspect in a robbery. No amount of pleading from the lady about her advanced pregnancy condition during the time of robbery and she just could not have committed the said crime with this condition, was heard by the police officer.  The Police officer did not have any discretion.  The system set up was such that once the AI face recognition identifies the suspect, Police are required to arrest her, bring her to the police station and book her.  

In this case, she was taken to the police station, booked and released on bail. Few days later the case against her was dismissed as the AI system has wrongly identified her.  It was also found out that she was not the first case and few more people, especially African American women were wrongly arrested and released later due to incorrect facial recognition model.

The speed in which the governments are moving on regulations and proliferation of AI tech companies delivering business application such as this facial recognition model demand urgent regulations.

May be citizens themselves should organize and let the people responsible for deploying these systems accountable.  The Chief of Police, may be the Mayor of the town and County officials who signed off this AI facial recognition system, should be made accountable.  May be the County should pay hefty fines and just not a simple oops, sorry.

Lots of attention need to be placed on training data.  Training data should represent all the diverse people in the country in sufficient samples.  Expected biases due to lack of sufficient diversity in training data must be anticipated and the model tweaked.  Most democratic countries have criminal justice system with a unwritten motto “Let 1000 criminals go free but not a single innocent person should go to jail”.  The burden of proof of guilt is always on the state.  However, we seem to have forgotten this when deploying these law enforcement systems.  The burden of proof with very high confidence levels and explainable AI human understandable reasoning, must be the basic approval criteria for these systems to be deployed.

The proposed EU act classifies these law enforcement systems as high risk and will be under the act.  Hopefully the EU act becomes a law soon and avoid this unfortunate violation of civil liberty and human rights.

More Later,

L Ravichandran

EU AI Regulations Update

Posted on August 18, 2023 - 5:30 am by

I have written some time back about EU AI Act draft circulation.  After more than 2 years, there is some more movement in making this a EU Law.  In June 2023,  the EU Parliament adapted the draft and a set of negotiating principles and the next step of discussions with member countries has started.  The EU officials are confident that this process will be completed by end of 2023 and this will become an EU law soon.  Like the old Hindi proverb “ Bhagawan Ghar mein Dher hain Andher Nahin”. Or “In God’s scheme of things, there may be delays but never darkness”.  EU has taken the first step and if this becomes a law by early 2024, it will be a big achievement.   I am sure USA and other large countries will follow soon.

The draft has more or less maintained its basic principles and structure. 

The basic objective of the new law is to make sure that AI systems used in the EU are safe, transparent, traceable, non-discriminatory and environmentally friendly.  In addition, there is an larger emphasis on AI systems should be overseen by people, rather than by automation alone.  The principle of proportionate regulations, the risk categorization of AI systems and the level of regulations appropriate to the risk are the central theme of the proposed laws.  In addition, there was no generative AI or ChatGPT like products when the original draft was developed in 2021 and hence additional regulations are added to address this large language models / Generative AI models. The draft also plans to establish a technology-neutral, uniform definition for AI that could be applied to future AI systems.

Just to recall from my earlier Blog, the risks are categorized  in to Limited risk, high risk and unacceptable risk.

The draft Law clearly defines systems which are categorized as “Unacceptable risk” and proposed to ban them from commercial launch within EU community countries.  Some examples are given below.

  • Any AI system which can change or manipulate Cognitive behaviour of  humans , especially vulnerable groups such as children, elderly etc.
  • Any AI system which classifies people based on various personal traits such as behaviour, socio-economic stataus or race and other personal characteristics.
  • Any AI system which does real-time and remote biometric identification systems, such as facial recognition which is usually without consent of the person targeted.   The law also clarifies that past data analysis for law enforcement purposes is acceptable with court orders.

The draft law is concerned about any negative impact on fundamental rights of EU citizens and any impact on personal safety.  These types of systems will be categorized as High Risk.

1)  Many products such as toys, automobiles, aviation products, medical devices etc. are already under existing U Product safety legislation.  Any AI systems that are used inside products already  regulated under this legislation will also be subjected to additional regulations as per High Risk category.


2)  Other AI systems falling into eight specific areas that will be classified as High Risk and required registration in an EU database and subjected to the new regulations.

The eight areas are: –

  1. Biometric identification and categorisation of natural persons
  2. Management and operation of critical infrastructure
  3. Education and vocational training
  4. Employment, worker management and access to self-employment
  5. Access to and enjoyment of essential private services and public services and benefits
  6. Law enforcement
  7. Migration, asylum and border control management
  8. Assistance in legal interpretation and application of the law.


Once these systems are registered in the EU database, they will be assessed by appropriate agencies for functionality, safety features, transparency, grievance mechanisms for appeal etc and will be given approvals before they are deployed in EU market.  All updates and new versions of these AI system will be subjected to similar scrutiny.  


Other AI systems not in the above two lists will be termed as “Limited risk” systems and subjected to self-regulations.  At the minimum, the law expects these systems to inform the users that they are indeed interacting with an AI system and provide options to change to a human operated system or discontinue using the system. 

As I have mentioned before, the proposed law is covering Generative AI systems also.  The law required these systems to disclose to the users that the output document or a output decision is generated or derived by a Generative AI system.  In addition, the system should publish the list of copyrighted training content used by the model.  I am not sure how practical this is given that ChatGPT like systems are reading every digital content in the web and now moving in to very audio / video content.  Even if the system produces this list which is expected to be very large, not sure current copy right laws are sufficient to address the use of this copyrighted material in a different form inside the deep learning neural networks. 

The proposed law also wants to ensure that the generative AI models are self-regulated enough not to generate illegal content or provide illegal advice to users.


 Indian Government is also looking at enacting AI regulations soon.  June 9th 2023 interview, Indian IT minister talked about this.  He emphasized the objective of “No harm” to citizen digital users.  Government’s approach to any regulation of AI will be thru the prism of “ User harm or derived user harm thru use of any AI technology”.  I am sure draft will be out soon and India also will have similar laws soon.

Let us discuss about what are the implications or consequences of this regulation among the various stakeholders.

  • AI system developer company ( Tech and Enterprises )


They need to educate all their AI development teams on these laws and ensure these systems are tested for compliance prior to commercial release.  Large enterprises may even ask large scale model developers like open.AI to indemnify them against any violations while using their APIs.  Internal legal counsels of both the tech companies and API user enterprises need to be trained on the new laws and get ready for contract negotiations.  Systems Integrators and outsourcers such as Tech Mahindra, TCS, Infosys etc. are also need to gear up for the challenge.  The liability will be passed down from the enterprise to the Systems Integrators and they need to ensure compliance is built in and also tested correctly with proper documentation.

  • Governments & Regulators

Government and regulatory bodies need to upskill their staff on the new laws and how to verify and test compliance for the commercial launch approval.  The tech companies are very big and throw in best technical as well as legal talent to justify their systems are compliant and if regulatory bodies are not skilled enough to verify then the law will become ineffective and will be only on paper.  This is a huge challenge for the government bodies. 

  • Legal community both public prosecutors, company legal counsels and defence lawyers

Are they ready for the avalanche of legal cases starting from regulatory approvals and appeals, ongoing copyright violations, privacy violations, inter company litigations of liability sharing between Tech, enterprise and Systems Integrators etc.

Massive upskillng and training is needed for even senior lawyers as issues arising from this law are very different.  The law degree curriculum needs to include a course on AI regulations. For example, the essence of a comedian talk show “learnt” by a deep learning model and stored deep in to neural networks.  Is it a copyright violation?   The model outputs similar style comedy speech by using the “essence” stored in neural network.  Is the output a copy right violation?  Who is responsible and accountable for an autonomous car accident?  Who is responsible for a factory accident, causing injury to a worker in a autonomous robot factory?  Lots of new legal challenges.

Most Indian Systems Integrators are investing large sums of money to reskill and also create new AI based service offerings.  Hope they are spending part of that investment in AI regulations and compliance. Otherwise, they run a risk of losing all the profits in few tricky legal challenges. 

More later

L Ravichandran

AI for Sustainability and Sustainability in AI

Posted on May 7, 2023 - 4:12 am by

I will be referring to the following 3 papers on this very interesting topic.

(1}  https://link.springer.com/article/10.1007/s43681-021-00043-6

 Sustainable AI: AI for sustainability and the sustainability of AI

A Van Wynsberghe – AI and Ethics, 2021 – Springe

(2) https://www.researchgate.net/publication/342763375_Carbontracker_Tracking_and_Predicting_the_Carbon_Footprint_of_Training_Deep_Learning_Models/link/5f0ef0f2a6fdcc3ed7083852/download

(3)      Lacoste, A., Luccioni, A., Schmidt, V., Dandres T.: Quantifying

the Carbon Emissions of Machine Learning. (2019)

While there is a tremendous push for using new-generation generative AI based on large language models to solve business applications, there are also voices of concern from experts in the community about the dangers and ethical consequences.  A lot has been written about this but one aspect which has not picked up sufficient traction, in my opinion, is Sustainable AI.  

In (1), Wynsberghe defines two disciplines on AI & sustainability.   AI for Sustainability and Sustainable AI.

AI for Sustainability is any business application using AIML technology to solve climate problems.  Use of this new generation technology to help in climate change and CO2 reductions.   Major applications are getting developed for optimal energy distribution across renewable and fossil energy sources. Any % extra use from renewable sources, help in less use of fossil fuels and help in climate change.  Various other applications may include better climate predictions and the use of less water, pesticides, and fertilizers for food production.  Many Industry 4.0 applications to build new smart factories, smart cities, and smart buildings fall into this category.

On the other hand, Sustainable AI measures the massive use of GPU and other computing, storage, and communications energy usage while building the AI models and suggest ways to reduce this.  While digital software development and testing can be done in a few developers’ laptops with minimal use of IT resources, the AIML software development life cycle calls for the use of massive training data and develop deep learning neural networks with multiple millions of nodes.   Some of the new generation Large Language models use billions of parameters beyond the imagination of all of us.  The energy use does not stop here.  Fine Tuning learning for specific domains or relearning is as energy-consuming or sometimes even more than original Training.   Some numbers mentioned in (1) are reproduced here to highlight the point.   One deep-learning NLP model consumed energy equivalent to 600,000 lbs of CO2.  Google Alpha-Go-Zero generated over 90 Tonnes of CO2 over 40 days it took for the initial training.  These numbers are large and at least call for review and discussions.   I hope I have been able to open your eyes and generate some interest in this new dimension of AI & Ethics i.e impact on climate change.

I am sure many of you will ask “Isn’t any next-generation industrialization from horse carriages to automobiles or steam engines to combustion always increased the use of energy and why do we need to worry about this for AI?”.  Or “There has been so much talk on how many light bulbs one can light for the same power used for a simple google search , why worry about this now ?”.  All valid questions.  

However, I will argue that

  1. The current climate change situation is already in a critical stage and any unplanned large-scale usage new of energy can become “the feather that broke the camel’s back!”.
  2. Use of fully data driven life cycle and billions of parameters, deep neural networks are being used for the first time at an industrial scale and industry-wide and there are too many unknowns.

What are the suggestions?

  • Energy consumption measurement and publication must become part of the AI & Ethics practice followed by all AI development organizations.   (2)  Carbon Tracker Tool and (3) Machine learning emission calculator are suggestions for this crucial measurement.  I strongly recommend organizations use their Quality & Metrics departments to research and agree on a measurement acceptable to all within each organization.  More research and discussions need to calculate the net increased use of energy compared to current IT tools to get the right measurement. In some cases, the current IT tools may be using legacy mainframes and expensive dedicated communication lines using up large amounts of energy and the net difference by using AIML may not be that large.
  • Predicting the energy use at the beginning of the AIML project life cycle also is required. (3). 
  • The prediction data of CO2 equivalent emissions need to be used as another cost in approving AIML projects.
  • Emission prediction also will force AIML developers to select the right size training data and use of right models for the application. Avoid the temptation of running the model on billions of data sets just because data is available!. Use the right tools for the right job.  You don’t need a tank to kill an ant!.
  • Ask the question of whether the use of deep learning is appropriate for this business application? For example, a simple HR application used for recruitment or employee loyalty prediction with Deep learning models may turn out to be too expensive in terms of Co2 emissions and need not be considered a viable project.
  • CEOs include this data in their Climate Change Initiatives Report to the Board and shareholders and also deduct carbon credits used up by these AIML applications in the company’s Carbon credit commitments.

More Later,

L Ravichandran

Small talk about Large Language Models

Posted on January 22, 2023 - 9:58 am by

Since its formal launch, ChatGPT has been receiving a lot of press and has also been the topic of – heated – discussions in the recent past.

I had played with generative AI some time back and also shared the result in one of my earlier posts.

Post ChatGPT, the investments in AI – or more specifically generative AI tech – based companies has seen a sharp rise.

There is also a general sense of fear – rising from uncertainty and the dread of the possibility of such technologies taking away specialized jobs and roles has been noticed across industries.

I was talking to an architect a few days ago and she said that in their community, the awe and fear of AI tech is unprecedented.

With just a few words, some of the sketches generated by tools like Dall-E, Craiyon, Stable diffusion etc are apparently so realistic and logical.. for example, when the query was to have the porch door opening out into the garden with a path to the main gate.. the image was generated in less than a couple of minutes..

With all the promise of creating new content quickly, many questions have also come up, without clear answers.

The first – also a topic of interest on aithougts.org – is that of ethics.

Whether it is deep fakes – btw, I had experimented with a technology that could have been used for this – when I was looking for tools to simplify podcast editing – on a platform called Descript – where I could train the model with my voice.. I had to read a predefined text for about 30 minutes – and then, based on written text, it could synthesize that text in my voice.. At that time, the technology was not yet as mature as today and so, I did not pursue.

I digress..

Getting back to the debate on generative AI, ethics of originality [I believe that there are now tools emerging that can check if content was generated by ChatGPT!] that could influence how students create their assignment papers.. or generate more marketing content, all based on content that is already available on the net – and ingested by the ChatGPT transformer.

Another aspect is the explainability of the generated content. The bias in the generated content or when there is a need for an expert opinion to also be factored in, would not be possible unless the source is known. The inherent bias in the training data is difficult to overcome as much of this is historical and if balanced data has not been captured or recorded in the past, would be very difficult to fix, or at least adjust the relevance.

The third aspect is about the ‘originality’ or ‘uniqueness’ of the generated content – let me use the term solution from now on..

There is a lot of work being done in these areas, some in research institutions and some in companies applying them in specific contexts.

I had an opportunity recently to have a conversation with the founder of a startup that is currently in stealth mode, working on a ‘domain aware, large language model based’ generative AI solution.

A very interesting conversation that touches upon many of the points as above.

 

You can listen to this conversation as a podcast in 2 parts here:

https://pm-powerconsulting.com/blog/the-potential-of-large-language-models-with-steven-aberle/

https://pm-powerconsulting.com/blog/episode-221/

 

Or watch the conversation as a video in 2 parts here:

https://www.youtube.com/watch?v=86fGLa9ljso

https://www.youtube.com/watch?v=f9DnDNUwFBs

 

Do share your comments and experiences with the emerging applications of GAN, Transformers etc.