Bootcamp Tutorial - Setting Up AI Workflow Files in ChatGPT & Claude Code Build w Agents - Aggregate Intellect - AI.SCIENCE 2026-09-26 | Bootcamp Tutorial - Setting Up AI Workflow Files in ChatGPT & Claude Code
The Importance of Privacy in ChatGPT and LLMs ML Explained - Aggregate Intellect - AI.SCIENCE 2023-09-05 | Summary -------Privacy is a crucial aspect to consider when it comes to chat GPT and Llms (large language models) in general. There are several reasons why privacy should be a top priority in these technologies. This essay discusses the importance of privacy in chat GPT and Llms, the legal obligations and market concerns related to privacy, incidents highlighting privacy concerns, measures taken by companies to protect user data, challenges in corporate environments, and the risks of re-identification when combining quasi-identifiers. The essay also includes a summary of Patricia Thaine's presentation and the subsequent Q&A session, where various topics related to privacy and Llms are discussed. Topics: ------- Legal obligations and market concerns * Legal obligations when handling customer data containing personal information * Data protection regulation compliance, such as GDPR * Market concerns and the impact of privacy on trust Privacy incidents and reputation * Data leaks and privacy concerns with OpenAI and GPT * Negative reputation resulting from not prioritizing privacy Measures to protect user data * Microsoft's Azure Open AI Services * Salesforce's Einstein GPT * Importance of removing or protecting sensitive data Challenges in corporate environments * Handling protected health information (PHI) without being a healthcare company * Compliance with regulations like HIPAA and PCI DSS Risks of re-identification * Differentiating between direct identifiers and quasi-identifiers * Examples of risks and studies on re-identification * Importance of effective de-identification techniques Discussion on privacy and Llms * Use of Llms offline for identification purposes * Severity of data breaches and the role of regulations * Corporations introducing practices to address privacy concerns * Challenges in relying solely on regulations * The evolving landscape of privacy and copyright in the AI industry
Building Ernie for Topic Modelling Using LLMs ML Explained - Aggregate Intellect - AI.SCIENCE 2023-09-05 | Summary -------Richie Youm, a data scientist at While Simple, shares the experiences of building a topic modeling model called Ernie. The goal of the Ernie model was to provide a smoother experience for clients and improve service level agreements. They explored techniques like LDA and BERTopic but faced challenges with inconsistent results and noisy data. They rebuilt the taxonomy using GPT models and developed an efficient routing system for improved customer service. The presentation also discusses the potential for an automated customer service system and the importance of data privacy and security. Topics: ------- Exploring GPT Models * GPT models were considered for direct classification but faced challenges with hallucination and domain knowledge * Prompt engineering on the ChatGPT API showed promising results but was not sufficient Rebuilding the Taxonomy * GPT was used as a tool to assist in taxonomy development * The existing taxonomy had limitations and GPT was used to address them Efficient Routing and Data Labeling for Improved Customer Service * Challenges faced in handling sensitive customer data and the need for efficient routing and data labeling * Development of a PI remover using Presidio and Transformers to anonymize sensitive data * Alan used for topic and subtopic taxonomy extraction * Bernie, an efficient routing system, built using DistilBERT models for topic and subtopic predictions Output and Examples of a Topic Modeling Project * Examples of the output of the topic modeling project highlighting the accuracy of predictions * Importance of context in improving predictions * High accuracy of subtopic predictions Potential for Automated Customer Service System * Discussion on the potential for the project to lead to an automated customer service system * Other projects being worked on, including a research AI project for trading * Possibility of adding knowledge base into a chatbot for better customer service Follow-up Questions and Formal Formats * Discussion on creating a tool that asks follow-up questions based on user input and system analysis * Interest in formal formats like ontologies and knowledge graphs * Discussions with the builder of tax GPT Data Privacy and Security * Importance of data privacy and measures taken to ensure it * Open-source library for LM Gateway and different levels of PII removal * Plan to open-source a PII remover Conclusion * Importance of improving the taxonomy and incorporating domain knowledge * Potential for automated customer service systems * Importance of data privacy and security * User feedback and potential use of formal formats in the future
The Business Impact and Challenges of Using Large Language Models ML Explained - Aggregate Intellect - AI.SCIENCE 2023-09-05 | Summary -------Suhas Pai shares his experience and insights on the business impact of using large language models (LLMs) and the challenges involved in taking prototypes to production. He discusses the impact of LLMs in the business world, the trade-offs in text summarization, challenges in the finance industry, and addresses audience questions. He emphasizes the need for careful consideration of how LLMs can add value to a company's products and services. Topics: ------- The Business Impact of LLMs * Success in launching products utilizing GPT-4 * Differences between smaller custom models and larger models like GPT-4 * Importance of considering how LLMs can add value to a company's products and services Balancing Trade-Offs in Text Summarization * Complexities of text summarization * Trade-offs in achieving effective and accurate summaries * Criteria to consider: relevance, specificity, structure, factuality, coherence, succinctness, and length * Challenges of extracting relevant details while removing redundant information * Need to balance coherence and succinctness * Limitations of language models in addressing trade-offs Challenges and Limitations in the Finance Industry * High threshold of trust in the finance industry * Avoiding hallucinations or mistakes * Difficulties in achieving a balance between different criteria in summarization * Limitations of GPT-4 in domain-specific knowledge and reasoning abilities * Importance of trust and accuracy in the finance industry Q&A and Additional Insights * Interplay between supervised fine-tuning and augmented generation * Importance of evaluating reasoning capabilities in language models * Approaches to address consistency in longer summarizations * Engagement with the foundation model * Challenges of cost and optimization in the summarization process
LLMs in Production - A Costly Dilemma! ML Explained - Aggregate Intellect - AI.SCIENCE 2023-07-10 | When deploying machine learning models in production, there are three properties that are commonly desired: generalization, evaluation, and cost-optimality. We conjecture that for machine learning models, it is impossible to optimize all three. I will talk about our framework for cost modeling and evaluation for large language models (LLMs) and present our LLMOps production pipeline.
Diving Into Document Question and Answering Systems with LLMs ML Explained - Aggregate Intellect - AI.SCIENCE 2023-07-10 | We’ve seen the demos on how to answer questions from a document, but how does it work behind the scenes? In this talk we’ll focus on how embeddings and LLMs work for information retrieval tasks, and how to measure their effectiveness. We’ll also describe some the UX challenges when exposing these natural language retrieval systems to end users.
Project Update - Building a Live Book ML Explained - Aggregate Intellect - AI.SCIENCE 2023-07-10 | In this talk we will present one of Aggregate Intellect community projects where we have been using Large Language Models to turn the transcripts of our community meetings into sections of an Open Book. We will also present a companion slack app that answers questions from this book and some other resources so that you don't have to read it.
High Performance Computing on Cloud for Generative AI ML Explained - Aggregate Intellect - AI.SCIENCE 2023-07-10 | Farshad will be discussing how GenAI is pushing the limits of HPC ML Cloud environments and what many companies are asking for from model providers. This talk will touch the very bottom of the stack and then skip to the model layer of the stack.
ThinkGPT: Agent and Chain of Thought techniques for LLMs ML Explained - Aggregate Intellect - AI.SCIENCE 2023-07-10 | ThinkGPT is a Python library designed to empower Large Language Models (LLMs) with advanced thinking and reasoning capabilities and to allow developers to create agents with LLMs. This talk will provide an overview of ThinkGPT's key features and how it addresses challenges such as limited context, one-shot reasoning, and decision-making. The library offers thinking building blocks like memory, self-refinement, knowledge compression, and natural language conditions. We will explore examples demonstrating ThinkGPT's applications, including teaching new languages to LLMs and enabling them to understand and generate code using new libraries. Additionally, we'll discuss the concept of generative agents and how they can be implemented with ThinkGPT. Join this talk to unlock the potential of ThinkGPT and leverage thinking abilities in your language models.
LLM Starter Pack: A Pragmatic Guide to Success with the Large Language Models ML Explained - Aggregate Intellect - AI.SCIENCE 2023-07-10 | Large Language Models (LLMs) have taken the world by storm. There is no AI conversation today that won't lead to "what about LLMs?". These models are great tools for solving some problems but can they be used everywhere and for all problems? When should we use them and which real-world problems are best solved without them? How do I pick my LLM? What is the best check list to have to make sure our machine-learning-powered products will be successful in the age of LLMs?
PandasAI - Talk to Your Data ML Explained - Aggregate Intellect - AI.SCIENCE 2023-07-10 | Pandas AI is a Python library that integrates generative artificial intelligence capabilities into Pandas, making dataframes conversational. With PandasAI, you can easily perform complex queries and plot chart in natural language. In this talk, Gabriele, the main author of this library will present his journey and demo some of the use cases.
Which Open-source LLM to Choose for Your Task? ML Explained - Aggregate Intellect - AI.SCIENCE 2023-07-10 | Dozens of open-source LLMs have pounced on the scene in the last several months. Can you actually replace chat-gpt/gpt-4 with them? What are they useful for and how can you use them in production? How do you select one from dozens of available models? What are LLM evaluation measures actually measuring? In this talk, we will explore these questions and try to provide a framework to resolve the build vs buy conundrum that many individuals and organizations find themselves in
Computational Thinking ML Explained - Aggregate Intellect - AI.SCIENCE 2023-06-14 | The latest advancements, like #ChatGPT, have transformed the way we interact with AI. It's not just an API or coding playground anymore. It's a whole new world of knowledge interfaces that have unleashed a wave of innovative ideas.🔗 The key is connectivity. These tools act as the glue that brings different elements together, allowing us to connect various generative AI systems, databases, calculators, and other tools going well beyond the constraints of individual tools. 💡 Think of it as a partnership. These systems empower us to think in collaboration with each other and with machines, both in terms of constructing knowledge and fostering creativity. We're still in the early stages of exploring their potential, but the possibilities are immense. It's like the early days of movies and television, where initially people simply recorded plays, but soon discovered the unique capabilities of the medium. 🌍 On a societal level, the introduction of LLMs has triggered a wide range of responses. Some view it with awe and reverence, while others express fear and caution. It's important to find a balance between caution and excitement. Rather than extreme reactions, a mix of both can lead to a healthier and more constructive approach.🎨 Creativity will be revolutionized. Just as tools like DALL·E sparked a burst of experimentation and prompt sharing, #LLMs have sparked a similar phase of exploration. After this initial phase, we'll enter a period of deep tinkering and understanding, followed by a burst of new creations that we haven't even imagined yet. Observing kids interact with these technolgies provide an interesting perspective, as they approach these with wonder and excitement, unburdened by preconceived notions.🔮 While it's difficult to make specific predictions, we can anticipate that LLMS will have ripple effects throughout society. It's crucial to observe how people respond to these technologies and how they shape our interactions with them. 🌟 As we continue to explore and push the boundaries, LLMS will undoubtedly make a significant impact on the projects we create and the way we invent. The initial hype will settle, allowing real-world projects to emerge and make a difference. Collaboration between humans with each other, and with machines will become more seamless, fostering a new era of creative possibilities.
Invest in Deep Tech like an Engineer - Deep Random Talks ML Explained - Aggregate Intellect - AI.SCIENCE 2023-06-13 | 📈 Let's take a look at one of the successful deep tech investments RiSC Capital made. Semantic Health, a company specializing in natural language processing of medical records, caught their attention. The company addressed a significant problem faced by hospitals, where teams of humans had to manually convert healthcare records into billable codes. Semantic Health's AI-powered solution streamlined this process, offering better accuracy and efficiency. The team, led by Nicola Sahar, MD, combined a strong science background with a Harvard Business School education. Their dedication and the technology's proven success made it a fantastic investment opportunity.🤝 As a founder seeking an investor, it's important to remember that the relationship extends beyond capital. Early-stage investors like to form long-term partnerships, providing support beyond just funding. To attract their attention, it's crucial to build rapport and show a willingness to collaborate on the business side. Founders who are coachable, open to guidance, and value a strong working relationship have an advantage in securing investments.🚩 When evaluating potential founders, investors look for certain qualities. Red flags such as arrogance, lack of trustworthiness, or hidden agendas can hinder partnerships. Trust is essential for a successful collaboration, as it fosters efficient decision-making and resource allocation. Ensuring a good fit between the investor and founder is vital for a harmonious and productive relationship.🌐 Looking ahead, the future of deep tech investments in Canada holds tremendous potential. While US/Europe currently leads in this area, Canada is poised for growth. As successful entrepreneurs reinvest in the deep tech industry, it will fuel further advancements and expand the ecosystem. Over time, we can expect to see a flourishing deep tech scene in Canada, similar to that of the United States and Europe.
Total Recall with NLP and LLMs - Deep Random Talks ML Explained - Aggregate Intellect - AI.SCIENCE 2023-06-12 | 🤔 Have you ever thought about the relationship between cognition, language, and creativity? It's a fascinating topic! Language serves as our interface to communicate our thoughts and explore cognitive abilities. It's like connecting the dots between different ideas and finding unique overlaps.💡 What's even more intriguing is how language technology has advanced over the years. Initially, sentiment analysis was the limit, but then Transformers revolutionized the field. Now, we have generatively pre-trained models with emergent multitask capabilities. Language is not just the interface of cognition but also becoming the interface of intelligence in machines.🗨️ Conversations play a vital role in the creative thinking process. When we discuss ideas with others, it becomes clearer what we're trying to convey, and their input can even change our perspective. Language technology, focused on understanding and generating text, aims to facilitate this creative activity.⚡ We now have the ability to convey information to machine learning models using language. However, we still have much to learn about the representations used by these models. Understanding these representations will be crucial as we move forward and treat these models as if they have human-like representations.💭 Exploring the possibilities of three-way interactions between humans and machines is where it gets truly interesting. By involving humans as tools within the language models' process, we can bridge the context gap. Machines can ask follow-up questions, seek further information, and act as mediators in conversations, enriching the overall communication.🔍 This human-machine collaboration can be invaluable, providing different perspectives and helping to refine ideas. It's like having a third brain involved, keeping track of the conversation's nuances and providing valuable insights.
Running LLMs in Your Environment ML Explained - Aggregate Intellect - AI.SCIENCE 2023-05-22 | see more slides, notes, and other material here: github.com/Aggregate-Intellect/practical-llms
Building with LLMs Using LangChain ML Explained - Aggregate Intellect - AI.SCIENCE 2023-05-22 | see more slides, notes, and other material here: github.com/Aggregate-Intellect/practical-llms
Building ResearchLLM: automated statistical research and interpretation ML Explained - Aggregate Intellect - AI.SCIENCE 2023-05-22 | see more slides, notes, and other material here: github.com/Aggregate-Intellect/practical-llms
Learning-free Controllable Text Generation for Debiasing ML Explained - Aggregate Intellect - AI.SCIENCE 2023-05-22 | see more slides, notes, and other material here: github.com/Aggregate-Intellect/practical-llms
ChatGPT-like application for construction of mathematical financial models ML Explained - Aggregate Intellect - AI.SCIENCE 2023-05-22 | see more slides, notes, and other material here: github.com/Aggregate-Intellect/practical-llms
Modern Innovations in Fine-Tuning Large Language Models ML Explained - Aggregate Intellect - AI.SCIENCE 2023-05-22 | see more slides, notes, and other material here: github.com/Aggregate-Intellect/practical-llms
Exploring the agency limits of todays LLMs ML Explained - Aggregate Intellect - AI.SCIENCE 2023-05-22 | see more slides, notes, and other material here: github.com/Aggregate-Intellect/practical-llms
Optimizing Large Language Models with Reinforcement Learning-Based Prompts ML Explained - Aggregate Intellect - AI.SCIENCE 2023-05-21 | see more slides, notes, and other material here: github.com/Aggregate-Intellect/practical-llmslinkedin.com/in/mingkaidengLarge language models (LLMs) are versatile and can perform tasks like summarization, code generation, sentiment analysis, dialogue, translation, and storytelling depending on the prompt.The wording of the prompt can significantly affect LLMs’ performance, making it challenging to find the best prompt for a given task. Two prompts with the same meaning can lead to different outcomes.Prompt optimization is a challenging problem due to the large number of candidates. One way to address it is to formulate it as a reinforcement learning problem. This allows for more effective identification of the best prompts.The reinforcement learning approach involves training a prompt policy to learn correlations between words and their underlying score or reward. It is a powerful way to optimize prompts for large language models.Optimized prompts for RL problems can perform better than human-written prompts, even if they don’t follow human language. This utility of RL prompts is important to understand.The optimized prompts can transfer well across models, and the reinforcement learning-based optimization allows for more effective identification of the best prompts. Careful optimization is key for large language models.I developed a framework that combines a smaller language model for word correlations and a larger model for tasks. It can perform few-shot text classification and unsupervised control text generation. #MachineLearning #NLPOptimized prompts for the framework are consistently among the best performers, unlike manual prompts, which can vary widely in performance. Check out my graph comparing their performance across different models. #AI #NLPShorter optimized prompts lead to faster model runs and lower costs. I found that optimized prompts trained on one model can also be applied to other models with similar or even better performance. #MachineLearning #OptimizationMy framework is better than human-written prompts at capturing how language models respond to prompts. See the graph comparing the performance of manual prompts vs. optimized prompts. #NLP #DataScienceI made sure to package my framework code well and make it easy to set up. You can find it on GitHub. For instance, running a test style transfer experiment requires only 51 lines of code. #OpenSource #PythonOptimized prompts from my framework can even turn negative sentences into more positive ones while preserving the original meaning. Want to see a demo? #AI #NLP #SentimentAnalysis
Expanding the Capabilities of Language Models with External Tools ML Explained - Aggregate Intellect - AI.SCIENCE 2023-05-21 | see more slides, notes, and other material here: github.com/Aggregate-Intellect/practical-llmslinkedin.com/in/piesauceThe field of NLP is rapidly evolving with new models, tools, and techniques being introduced regularly. In fact, 90% of the content in the presentation did not exist a few months ago, and the content about LangChain and LlamaIndex is set to become woefully outdated within a month or two because those libraries are coming out with so many new updates. GPT-4 is evidence of the fast pace of development in the field. #NLP #GPT4 #MachineLearningInstruction tuning is a powerful technique for improving LLMs and making them more aligned with human preferences. OpenAI’s Instructor-GPT paper introduced the use of reinforcement learning and human feedback to align LMs/LLMs with human preferences. #InstructionTuning #LLMs #ReinforcementLearningEvaluations are ongoing to determine the full capabilities of LLMs. The speaker notes that LLMs are good at following instructions and query understanding, but their limitations are not fully understood yet, especially in their reasoning capabilities. #LLMs #Evaluations #MachineLearningBeing 99% accurate is not enough for many applications, especially if the failure cases are unpredictable. This is true for self-driving cars and may also be the case for a wide variety of NLP applications. #MachineLearning #NLP #Accuracy #selfdrivingcars Language models break down complex questions into manageable components. For example, they can answer “who was the CTO of Apple when its share price was lowest in the last 10 years?” by finding the date of the lowest share price and the CTO on that date.LLMs can filter out irrelevant results and automate tasks like flight searches. They call external APIs and databases and then synthesize coherent answers based on the output.LLMs can answer questions about a company’s policies, product planning, or any other information stored in Google Docs or Notion by using the data connectors provided by Lama index.LLMs can ensure answers don’t contain personal identifiable information or misinformation using a moderation chain. Making an LLM good at a particular domain is exciting, as passing exams in a particular domain is easy due to data contamination.
Leveraging Language Models for Training Data Generation and Tool Learning ML Explained - Aggregate Intellect - AI.SCIENCE 2023-03-22 | see more slides, notes, and other material here: github.com/Aggregate-Intellect/practical-llmslinkedin.com/in/gordon-gibson-874b3130** Large Language Models and Synthetic DataResearch on using unlabeled data to improve large language models is exciting, and the potential impact on natural language processing is vast. These models are changing the way we think about language and the possibilities of AI.Large language models are trained on vast amounts of unlabeled data in a self-supervised manner. They continue to show impressive results as they scale, producing higher quality and more human-like text even for tasks they are not explicitly trained to perform.As AI adoption increases, there will be a growing demand for human annotators soon surpassing human capacity to keep up with data needs of increasingly larger models and more complex use cases.One of the interesting new areas is to use large language models themselves to create new data for training. For example, synthetic data can be generated to augment existing datasets for improving LLMs themselves or other types of models.These kinds of data augmentation techniques can be used to improve large language models reducing the need for human annotation. This can reserve the more expensive human labor for creating high-quality or mission critical datasets.Another trend we're seeing in the industry is that human annotations will be used more for creating evaluation or quality control datasets, while LLMs will be used for generating training data. #machinelearning #datageneration #humansintheLoopThis approach combines the strengths of both human annotation and machine learning, and has the potential to increase research capacity by generating more training data. #machinelearning #datageneration #humansintheLoop #researchcapacity** Using Large Language Models for Data GenerationRecent research papers have shown that we can use large language models to generate weak labels for tasks such as named entity recognition, sentiment analysis, and question answering. We can then have humans revise or validate these labels to create high-quality training data. #machinelearning #datageneration #humansintheLoopToolformer is one example of a system that uses LLMs to generate data for training other models. It splits up the data set and samples API calls to generate possible inputs and outputs for different tools. Toolformer then computes the model's loss to predict the next words in the sequence.** Techniques for Filtering Data for LLM Fine-Tuning... see more notes on the link above** Fine-Tuning Language Models with Self-ConsistencySelf-instruct and self-consistency approaches are suitable for fine-tuning with available (frozen model) endpoints. These approaches involve generating new tasks and instructions for the model to fine-tune on. ???Self-instruct papers use human-created examples to train models to generate instructions and outputs for tasks. Language models can also use self-consistency to fine-tune themselves by generating different outputs and comparing them to select the most frequent one.This technique does not require the model to know the ground truth, but as the models become larger, the most frequent output is often the correct one. It is observed in literature that larger language models generate more accurate responses.The model filters down data using self-consistency, and if the majority of generations produce a specific output, e.g., "nine," the model takes all the cases where "nine" was generated as the output, assuming that these are correct, and feeds them back into the model which creates a feedback loop that improves the model's performance over time.** Reinforcement learning from AI feedback (RLAIF) for Harmless and Helpful Language ModelsRLAIF is a promising application for large language models where models can learn from their own mistakes and improve over time. It is a method for training language models to be more helpful and harmless. It uses a Constitution to critique the model and train it to rank outputs based on preferences.To train the model, harmful responses are generated through red teaming requests, and the Constitution is used to guide the model's behavior and critique its responses. The model is then fine-tuned on a dataset of revisions based on its critiques. #RedTeaming #ModelTrainingThe Constitution is created by humans as a guideline for the model's behavior, but the model is able to critique itself and generate revisions based on the Constitution. This allows for more training data to be generated using the model itself, increasing research capacity. #AIResearch
Generative AI: Ethics, Accessibility, Legal Risk Mitigation ML Explained - Aggregate Intellect - AI.SCIENCE 2023-03-22 | see more slides, notes, and other material here: github.com/Aggregate-Intellect/practical-llmslinkedin.com/in/noelleai** Use of generative AI to improve accessibility and the lives of people with disabilities... see the link above for more notes** Managing and moderating LLMs to reduce bias, and increase fairness6/20: Deliberate choices that companies make can reduce the impact of bias, and mitigate risks. Mitigating potential issues with LLMs requires a comprehensive approach, including diverse datasets, fine-tuning foundation models, utilizing hardware resources, managing content, and continuous monitoring.7/20: LLMs are trained on data sets generated by humans, and the awareness of potential sources of bias and inaccuracies in that data is crucial. Diverse datasets are necessary when training LLMs to reduce the impact of those imperfections.8/20: LLMs make assumptions based on training data. This can result in incorrect conclusions. Noelle mentioned that a falsely attributed scholarly article was taken as true by an LLM because it was making assumptions based on the training data.9/20: There is a need for inclusive data collection to ensure that AI solutions represent end users. Inclusive data collection along with keeping track of lineage and context can help mitigate biases that may be present in training data.10/20: LLMs can amplify bias over time, but there are ways to mitigate its impact. Having a human-in-the-loop process to monitor, manage and control the LLMs is crucial. #AmplificationOfBias #HumanInLoop11/20: LLMs can speed up the process of generating content, but if not managed at scale, it can slow down the process by generating inappropriate or poor-quality content. Moderation and management of the content generated by LLMs are crucial. #ContentGeneration #Moderation12/20: Specific performance metrics for LLMs are essential to understand what is a good model and what is not. Continuous monitoring by a human team is necessary to ensure that the model is working correctly. #PerformanceMetrics #ContinuousMonitoring** Versatility and scalability of potential LLM use cases13/20: LLMs are built on foundation models that can be fine-tuned to fit specific business needs. This allows businesses to create LLMs tailored to their specific needs and goals, making them increasingly popular with companies willing to invest in them.14/20: The demand for LLMs is growing rapidly, and ML developers need to be able to build and deploy them quickly to meet the demand. This puts a premium on rapid development and deployment processes.15/20: LLMs can do more than just conversation. They can generate natural language requests and responses from various sources, including customer signals, website data, and ticketing systems. They can even be used to automatically formulate human questions and generate Power BI dashboards.16/20: Next generations of LLMs might be multi-modal which means that they can combine different types of input, such as images and text, to generate output. This makes them valuable in a variety of contexts and use cases.17/20: LLMs require significant hardware resources to operate, and only a few labs in the world can support the required hardware. A balance has to be made between building internally, using offerings from mainstream vendors, and ones from emergent providers.** Challenges and Risks of LLMs18/20: Organizations have a responsibility to guard against potential legal issues when using LLMs. Noelle emphasized the responsibility of organizations to think through potential legal issues and approach projects with an awareness of the risks involved.19/20: Evaluating risks and mitigating them in the solution is important. There are use cases for LLMs that are relatively easy to approach and mitigate risks, such as customer call centers and customer service ticketing. However, more rigor and discipline are required for projects using codex, which were trained on Github repos.20/20: Indemnification is important when using LLMs to protect against ownership challenges that may arise in the future. Enterprise level solutions provide more indemnification than research models like Dalle, especially if the model is not custom trained on your own art.
Incorporating Large Language Models into Enterprise Analytics ML Explained - Aggregate Intellect - AI.SCIENCE 2023-03-22 | see more slides, notes, and other material here: github.com/Aggregate-Intellect/practical-llmslinkedin.com/in/rajistics** Advantages and Use Cases of Large Language Models1/14: NLP use cases in the enterprise primarily focus on text classification, summarization, question answering, and embeddings. Large language models like GPT-3 and ChatGPT can transform how enterprises approach these tasks.2/14: LLMs offer advantages over traditional NLP models, such as better performance in classification and language generation for use cases like financial sentiment analysis and customer service bots.3/14: To leverage LLMs in enterprise workflows, companies can start by exploring pre-trained models, fine-tuning them on their data. However, LLMs are general-purpose models and adding domain-specific information might be challenging. Using dedicated models for certain tasks, combined with LLMs can be more efficient and cost-effective.4/14: LLMs can connect previously siloed domains in an enterprise, allowing for more efficient and natural language searches across multiple domains. This makes it easier to access relevant information and insights from different parts of the organization.5/14: The UI is just as important as the model in solving business problems. Therefore, it's essential to focus on UI/UX design in addition to developing models. While models can add new capabilities, the user interface plays a critical role in presenting them in a valuable way.** Interfacing LLMs and other Enterprise Systems6/14: LLMs can be decision-makers, but it's crucial to set them up for success by combining them with other tools. For instance, for a math question LLM could be set up to call a calculator and then show the result to prevent it from giving the wrong answer with confidence.7/14: Large Language Models can enhance information retrieval by combining classic data / knowledge bases with LLMs to provide more accurate and human-readable responses to queries. For example, LLMs were used to retrieve information on how to install Transformers from Huggingface documentation with citation to resource.** Choosing and Keeping Up with LLMs8/14: Choosing the right LLM depends on the specific use case, and there are various options available with different licensing requirements and costs.9/14: When choosing an LLM, consider factors like availability, cost, and accuracy for the specific task at hand. Open source models like Hugging Face can provide a wide variety of models and tools leveraging the collective knowledge of the community.10/14: The field of LLMs is evolving rapidly. Staying up to date on the latest developments and trends is crucial to make the most of this technology. Keep learning to keep improving!11/14: For those looking to learn AI, OpenAI Sandbox and Hugging Face Spaces are excellent starting points. OpenAI Sandbox provides templates and examples for learning AI, while Hugging Face is a useful resource for natural language processing. LangChain and LlamaIndex are also good examples to get started.** Democratization and Ethical Considerations in AI12/14: The democratization of AI is crucial for its widespread adoption. Co-pilots and natural language interfaces are enabling more people to experiment with AI, even those without a technical background. Lowering the barriers to entry for AI can have a significant impact on the democratization of AI.13/14: Ethical considerations are essential in the development and deployment of AI. It's crucial to include diverse voices in the design and decision-making processes, educate users on the potential dangers of generative AI, and consider the ethical implications of AI.14/14: Bias is a significant challenge in AI. Historic data used for training models can perpetuate biases and historical injustices. Therefore, it's crucial to be aware of these biases and work towards minimizing their impact.
Integrating LLMs into Your Product: Considerations and Best Practices ML Explained - Aggregate Intellect - AI.SCIENCE 2023-03-22 | see more slides, notes, and other material here: github.com/Aggregate-Intellect/practical-llmslinkedin.com/in/denyslinkov** Considerations for adopting LLMs1/16: Running large language models in house can be costly, which is why many people use APIs. It's important to ensure your use case has a good ROI to avoid wasting resources.2/16: When deciding how much to invest in using LLMs, companies need to consider their bet size (level of investment) and company size (effort required to adopt LLMs). This should be done with a careful assessment of consequences on their existing customers and their product development roadmap.3/16: Therefore companies need to understand their business and customers before adopting LLMs. Voiceflow, a conversational AI platform used by various verticals, experimented with LLMs and how they should be combined with existing capabilities.4/16: Even large companies, eg. Google's Bard and Bing's demos have had instances where LLMs generated incorrect information damaging their brands. Having humans in the loop, domain experts when appropriate, is important to ensure accuracy.5/16: UX and UI are critical for LLM adoption. Incorporating fun and natural features, such as shortcuts and accepting / declining recommendations, are important for making LLMs easy to use and understand.6/16: Human-in-the-loop performance improvements at creation time are less risky than having that kind of feature in run time. One should carefully consider the trade offs when implementing user feedback loops.7/16: Caching might be a good way to reduce cost and improve performance. Though it has to be done in a way that context for different users is handled properly to ensure the best experience.** Deployment methodology for LLMs8/16: When deploying large language models, there are different options to consider. Using a pre-trained model service is easier, but building and hosting your own solution gives you more control.9/16: Building a product that generates revenue is essential, but investing time in a minimally viable platform is also important. Neglecting the platform can lead to technical debt and slow down the iteration process for new models. There should be a balance in effort put in model, product, and platform development.** Testing and Fine-Tuning for LLMs10/16: When deploying LLMs in production, testing is crucial. Denys Linkov built a test suite to check if prompts work, processing the code, running python, and documenting errors. This is on top of initial manual testing and collecting errors from the data warehouse.11/16: For example, since the output of Open AI API is probabilistic it might return poorly formatted JSON responses. This at inference time might be easy to deal with but at high volume could cause unforeseen failures in production systems.12/16: Fine-tuning can solve formatting issues, especially for few-shot learning including desired output, examples, and chain of thought reasoning.** Considerations for Third-Party Providers13/16: Running a product with LLMs comes with its own set of challenges, like unpredictable response formatting, unreliable uptime, or other upstream dependencies.14/16: When choosing NLP provider APIs or open-source libraries, it's crucial to consider their reliability and ease of integration. Companies can opt for a single or multiple NLP provider APIs based on their required uptime.15/16: While OpenAI is perceived to have the best models, other providers may offer more reliability. The trade-off between model accuracy and uptime is a critical consideration for companies depending on their service requirements.16/16: With LLM space moving quickly, it might also become easier to deploy own models instead of relying on third-party providers at least for some components of the system.
Commercializing LLMs: Lessons and Ideas for Agile Innovation ML Explained - Aggregate Intellect - AI.SCIENCE 2023-03-22 | see more slides, notes, and other material here: github.com/Aggregate-Intellect/practical-llmslinkedin.com/in/josh-seltzer... (see more notes on the link above)** IP strategy for large language models6/21: When building large language models for commercialization, it's important to consider IP strategy and seek advice from experts.7/21: Applying large language models to improve existing capabilities may not be patentable, but rethinking the entire approach to solve a problem in a new way is more likely to be patentable. #MLdevelopment #IPstrategy** Large language models for R&D8/21: During R&D, specify tasks, models to optimize, & create dataset to evaluate models against. Use large language models for weak label generation & data augmentation to make data set curation easier. #AI #ML #R&D9/21: R&D conversations are often open & difficult to structure, but they can still be turned into components used by large language models. For example, to create a conversational bot for Slack, modularize into 3 LLM calls for message, trigger, & audience classification. #AI #ML #chatbot10/21: With modularization & templated code repository, create conversational bot very quickly. Use LLM-generated info for message content classification, trigger classification, & audience classification. #AI #ML #chatbot #Slack** Microservices and modularization of products based on large language models11/21: Large language models (LLMs) can be used in various stages of a project, including engineering and production. They can be leveraged to build microservices or components that work together to produce the desired output.12/21: Treating a LLM as one of the microservices of a product enables breaking down the problem into smaller and more manageable pieces. This allows for easier implementation and development, additional R&D, unit testing, and quality control.13/21: This approach also allows for easier explainability and optimization of the process. The components in the architecture of a system that leverages LLMs should interface the language skills of GPT-like models and domain-specific expertise to get the best results.14/21: In our slack app example, the LLM prompts used depend on the context and specifics of what is being done. In some cases, having a good prompt engineer is crucial, while in others, we can just pass exemplars that would have a bigger influence on the performance.15/21: We can add a dialogue analysis component to control the prompts generated by LLMs in a particular domain. This component can infer information necessary to determine the most relevant examples when generating probing questions.16/21: Another component can create dynamic prompts for in-context learning by choosing the best exemplars from a repository or by retrieving documents containing domain knowledge. This can significantly improve the performance of LLMs.17/21: Finally, a quality control component can rank generated candidates in case the LLM generates a question or answer that is not suitable. Rule-based and human-crafted questions generated using question recipes can also be used to ensure that inappropriate responses are avoided.** Production stage of large language models18/21: In production, large language models can be optimized by combining them with other components to create an ecosystem of microservices that work together. This approach ensures better performance and improved efficiency of the overall system.19/21: When optimizing costs during a project, evaluating both performance and cost is crucial. If a cheaper model performs almost as well as a more expensive one, it's advisable to choose the former. This way, you can save money without compromising on performance.20/21: After some data is collected and the overall system performance is more well established, it's sometimes possible to replace large language models with cheaper and leaner models in production. This approach helps to reduce costs without sacrificing performance.21/21: When building a startup with large language models, it's essential to consider the service level availability uptime of cloud infrastructure. Microsoft Azure Open AI service offers 99.9% uptime, ensuring that your system will be available to users when they need it.
The Emergence of KnowledgeOps ML Explained - Aggregate Intellect - AI.SCIENCE 2023-03-22 | see more slides, notes, and other material here: github.com/Aggregate-Intellect/practical-llmslinkedin.com/in/amirfzpr** KnowledgeOps and Development Processes1/14: #KnowledgeOps is about managing an organization's or community’s knowledge assets and processes to enable reuse and collaboration. #DevOps and #ModelOps (#MLOps) are specific examples that help develop software faster and with lower chances of failure.2/14: #ShiftingLeft and #CI/CD are important #DevOps concepts that bring testing and quality assurance to the beginning of the software development process and provide tools to automate them for consistency and repeatability.3/14: Although automation has improved many parts of the development process, the step of discovering and planning is still largely manual, requiring continuous communication and collaboration with teammates, which creates a single point of failure.4/14: #ModelOps (e.g., #MLOps) has allowed for the automation of data and model handling, but the manual interpretation and decision-making steps are still present. #Automation** Generative AI, and Continuous Exploration / Continuous Integration / Continuous Delivery5/14: Generative AI can help us shift further left into the exploration, planning, and coding steps, significantly improving our ability to explore options and conduct experiments. #GenerativeAI #SoftwareDevelopment6/14: There will eventually be more automation in our problem-solving processes, but in the meantime, tools built with generative AI will significantly augment our ability for interpretation and decision-making in ensuring the success of the development of complex software systems. #MachineLearning #AI7/14: Emerging tools for thinking allow for a more experimental approach to knowledge-intensive work, allowing for continuous hypothesis generation and experimentation leading to CE/CI/CD. #ContinuousExploration #ContinuousIntegration #ContinuousDelivery8/14: There is a trend towards interfacing generative copilots, retrieval systems, and other knowledge-intensive systems to create thinking machines with memory and reasoning skills. #GenerativeAI #RetrievalSystems #Reasoning9/14: Language models are essential for these tools because they need to interpret users’ instructions usually provided in natural language, communicate results, and facilitate human-human communication. #LanguageModels #Communication10/14: Language models can give us the ability to articulate and communicate complex ideas effectively to stakeholders and team members enabling more efficient problem solving in communities and organizations. #LanguageModels #Communication** Adoption of Knowledge-Ops11/14: In bigger companies, beyond technology, the biggest barriers to implementing #KnowledgeOps are cultural problems; eg. political reasons that prevent a unified and integrated knowledge and expertise system connected to knowledge bases of all teams across the enterprise. #CulturalProblems12/14: Since all #GPT can reliably provide in short term is the language skill, primarily NLU/NLG, smaller language models trained on internal knowledge can be built to avoid privacy and data access issues. #NaturalLanguageUnderstanding #NaturalLanguageGeneration13/14: Adoption of #LLM enabled thinking tools will start in smaller companies, and with improvements in corporate culture and maturing technology, we will see bigger companies joining the movement.14/14: With these tools being able to talk to us, remember our context, and reason about the world around us without the barriers of coding and formal language, we can accelerate #KnowledgeOps to the point where no idea is too expensive to try. #Automation
Neural Search for Augmented Decision Making - Zeta Alpha - DRT S2E17 ML Explained - Aggregate Intellect - AI.SCIENCE 2023-02-28 | 01:54 where did Jakub's journey begin03:21 NLP in the 90's04:19 Jakub's Resume analysis company06:54 Selling TextKernel and starting Zeta Alpha09:44 Initial assumption: deep learning researchers need a tool to keep track of advances in their field 11:27 Putting together a team to build a technical knowledge management product15:31 Rise of large language models, augmented decision making, and enterprise search 20:07 Does finding documents for people (search) solve the problem that users have?22:37 Reading papers is a nightmare (and how GPT3 et al can start to tackle this)25:38 Knowledge management as a human-machine-interface problem29:47 Differences between what academic and industrial audience would want32:49 Multi-task nature of large language models to rescue?35:49 Are language models alone enough?39:22 What's down the pipe for Zeta Alpha?41:12 Verdicts and wrap upKEY TAKEAWAYS1: Jakub is an NLP expert who's been working on neural information retrieval since the early 90s using probabilistic machine learning methods. He founded Textkernel, a resume analysis company that used machine learning on unstructured data before switching to large neural models like LSTMs in 2015-2016 before he exited in 2019.2: Jakub's new company, Zeta Alpha, aims to connect people with knowledge in general. They provide a platform for discovering new papers, organizing knowledge, and sharing notes with colleagues. The initial audience was deep learning researchers, but the platform is useful for anyone in a fragmented world of scientific discovery who wants to avoid reinventing the wheel.3: Neural search is at the core of using AI for better decision-making, as it enables people to discover knowledge and information easier. In the last few years, there has been significant progress in neural search, with accuracies of relevance of search tripled in some domains.4: Transformer-based language representations have bridged the gap of natural language search and limited keyword-based search achieving performances that are actually useful even when people don't know what they are searching for; especially important for researchers or decision-makers.5: Neural search can embed a query in a vector space using large language models and connect it easily with passages of text or entities like people using algos like approximate nearest neighbor search. Personalized recommendations based on the user's reading history can help people make better sense of search results.6: To improve user adoption in enterprise, the workflow needs to be turned around to bring the information to people where they usually sit in tools like email or Slack, rather than having them go to a search engine and type all kinds of queries. As an ML developer, it's important to consider the user workflow and interaction with the system.7: Collaboration between machines and humans is necessary for users to trust the answers from the system. Transparency is crucial, and traceability to documents is important for research applications. Language models are evolving to incorporate retrieval components for efficient indexing and referencing back to original material.8: Advanced academic work is often done by large tech companies, and open-source data sets are available for building foundation layers for applications. Achieving high accuracy for specific domains or languages may require closing performance gaps and using generative models to create synthetic training data.9: Large language models can capture syntax very well, but interfacing with other data structures like graphs may be necessary in niche technical areas or where there are complicated semantic relationships. Some large language models can even translate intents/questions into actual programs (e.g., SQL queries or Python scripts) to facilitate that. 10: Building and maintaining a large knowledge representation may be too costly for most companies, and progress in machine learning is fast enough to make those investments unnecessary. There will be companies that handle publicly available knowledge as a foundational layer with abilities to fine tune to each client's needs. 11: It is exciting to see an ecosystem of products evolving that are reimagining knowledge work, representation of knowledge, and intellectual property. Changing the scientific publishing system is challenging since it is tied to other aspects of the economy, but there are companies in the AI and decentralized science space that are making a difference.12: Knowledge is a shared and communal commodity, and any solution that doesn't involve grassroots and community-focused efforts will be incomplete. Any platform in this space needs to foster a sense of community and collaboration, augmented by AI and other technologies.
Distributed Data Engineering for Science - OpSci - Holonym - DRT S2E16 ML Explained - Aggregate Intellect - AI.SCIENCE 2023-02-21 | 02:10 Our favorite parts of brain06:11 What does Shady (OpSci) do?08:19 Shady's PhD work and the inspiration for a platform for data engineering for scientists 11:16 GitHub for complex datasets?15:32 What's wrong with how scientific data sharing and handling works now and what can be done about it?19:27 Large scale lab automation, and epiphany: web 3, as an ongoing consensus machine, could be a way to tackle this problem24:47 DAOs, permissionless systems, and the hell they raise!30:26 What are zero-knowledge proofs and why do they matter?34:06 What's the use case for zero-knowledge proof in the context of using DAOs for scientists working together? Birth of Holonym35:29 Is identity an important component of crowd-based scientific discourse and how is that different from social media?38:06 How do you ensure identities persist across platforms and tools? 41:11 Going from solving a problem for yourself, to making a movement43:21 Biggest next challenges? Holonym as the base layer for scientific data engineering and decentralizing the development 46:49 Wrap up and verdictsKEY TAKEAWAYS:1. The inspiration for OpSci came from a longitudinal neuroimaging study that tracked the brain development of children from ages 11 to 13 into adulthood. The study involved collecting and collaborating on complex datasets across institutions.2. The study had to use cloud-based environment that raised privacy and regulatory concerns. The brain hack community, a decentralized science community, was instrumental in finding a solution using the interplanetary file system (IPFS) to share datasets without going through a third-party server.3. However the need for a tool that can version control datasets, which are stored in all sorts of different representations, was still withstanding. Projects like Git-Annex (managing large files with git, without storing the file contents in git) and Datalad (Open-source distributed data management) allow for binary version control and streamline the publication of scientific experiments to a distributed archive. 4. Permissionless systems, like Datalad, have the potential to create open ecosystems but at the cost of potential adversarial behavior, particularly through automation such as bots. This presents a challenge for ensuring the security and trustworthiness of datasets stored on blockchain networks. Therefore it is important to verify the identities of participants in data commons or marketplaces.5. The delicate balance between privacy and security in permissionless blockchain networks requires a solution that enables registration of accounts on chain while maintaining anonymity and privacy. The solution involves creating a smart contract that represents elements of the identity of the individual and allows any agents to link other cryptographic key pairs to the smart contract.6. Zero-knowledge proofs (ZKPs) can preserve anonymity while uplifting permissionless systems, making it possible to produce proof of X without revealing any information about X. This technology is particularly useful in solving regulatory compliance issues, such as the problem of non-US residents participating in a US-based lobbying system.7. Anonymous accounts on platforms like Twitter and YouTube are often used as throwaway accounts with no repercussions or penalties. Blockchain technology can provide an immutable solution with gating mechanisms for discussion, which can link accounts to publication history, peer reviews, and future matching, creating a character or reputation in the scientific world.8. There is a challenge in ensuring the continuity of an identity, which leads to the problem of civil resistance. Providing proof of uniqueness is crucial, and this can be achieved through generating a checksum of supporting records such as a driver's license or diploma, which is then encrypted and put on the blockchain. The blockchain provides a single source for reference, which eliminates the possibility of generating an identical proof and associating it with a new identity.9. Scientists are often pressed for delivery, which makes it challenging to convince them to use new technologies or methods. Building strong relationships and having a compelling reason to use new tools are key to adoption.10. The emergence of decentralized, web3-native tools and protocols is changing the way science is conducted, allowing for unprecedented funding and collaboration opportunities, and a chance for us to reimagine how science is done.11. Holonym, a platform that aims to link on-chain records with off-chain verifiable impacts, has the potential to revolutionize how scientists publish their work and seek funding. Identity and privacy are crucial considerations in this new ecosystem.
Data Products - Accumulation of Imperfect Actions Towards a Focused Goal - DRT S2E15 ML Explained - Aggregate Intellect - AI.SCIENCE 2023-02-14 | 03:17 Brian's career as musician, designer, and product consultant10:48 What is the process to uncover desirability, feasibility, and other product requirements17:59 Importance of defining success quantitatively as early as possible20:06 How do you deal with interviewing end users, if access to them is complicated 27:27 What does lo-fi prototype mean for algorithms? ("so what" exercise)32:28 How do you deal with stakeholders who are married to a solution instead of focusing on problem solving? 36:17 Data centric ML, and interpretation of data explorations, and ML results45:09 Human centric data science, data assumptions, context, and provenance46:29 Wrap up and verdictsKEY TAKEAWAYS1. The process of creating a robust user experience starts by considering all stakeholders and understanding the end-user's needs 2. Regardless of your organizational role, you can demonstrate product leadership by asking the right questions to surface unspoken needs.3. Product development has to be focused on driving value and outcome, rather than just delivering outputs. No one needs a technically right, and effective wrong solution.4. Product development process has to be lean and focused on regular build-measure-learn cycles that maximize traction (how many new user showed concrete intent to use the product vs how many interviews we did)5. Cross-functional teams need to own the problem space and work collaboratively to find the best solution, including considering whether or not machine learning is necessary for the desired outcome.6. The "So What" exercise (aka "abstraction laddering") is a valuable tool in figuring out the fundamental reason you are solving a problem, as it helps you think about the overall system design based on users' workflow, and how an algorithm can improve that process.7. It is important to tackle the riskiest assumptions first using experiment artifacts like low fidelity mockups, clickable prototypes, and eventually live data "steel threads".8. The operationalization of the model must be included in the design of the solution and is not a separate step. The end-to-end experience is what makes the solution a complete data product, and not just a project.9. When designing decision support tools, eg. based on advanced analytics, focusing on a UX that conveys trust is very important. Frameworks like CED (Conclusions, Evidence, Data) are good options to consider for that purpose
Unfolding the Maze of Funding Deep Tech; Metafold - DRT S2E14 - Ft. Moien Giashi, Alissa ross ML Explained - Aggregate Intellect - AI.SCIENCE 2023-02-07 | 03:15 Elissa's transition from PhD in Mathematics to becoming a Geometry consultant04:24 Wait! What's Geometry consulting?05:13 What problem does Metafold solve?06:42 1-min Metafold pitch!07:49 The decision process from consulting to product business09:50 Focusing on a particular use case; transition from hardware + software to only software (and surprise! Covid's impact)11:38 Elissa's take on what deep tech means12:47 Moien's take on what deep tech means13:46 Go-to-market strategy and business models for deep tech15:45 Can you bootstrap a deep tech company?16:58 Assumptions deep tech founders make about how they should pitch to investors 18:30 A Pitch to an investor has to tell the story of how both the company and the investor are going to make a lot of money21:28 What does it mean with investors say "no" to your briliant idea (hint: not much)22:30 Founder - investor fit24:53 How can deep tech founders learn how to talk to investors?27:04 How to build confidence to talk about business stuff with investors as a very technical person30:43 Moien's take on how pitch deck should be structured (and bizzare relationship deep tech founders have with money)35:37 Do accelerators help deep tech founders?37:48 At some point you need capital not more mentorship39:03 How will previous experiences guide Elissa's next fundraising round40:55 Amir's rant on funnels and systems for fundraising41:21 Wrap up and verdictsKEY TAKEAWAYS:1. Metafold is a software company that provides design software for additive manufacturing to make it easier and faster to get complex geometry products to market.2. It originated from a geometry consulting firm as a hardware company, but pivoted to a software product driven by the desire to have a broader impact to unblock engineers from making progress with 3D printing innovations.3. The process of raising funds for the startup involves navigating the pressure from investors to have a narrow focus on a single use case for the technology while staying true to a grand world-changing mission. 4. "Deep technology" solutions have significant R&D risk associated with them in addition to the usual business and market risks "shallow technology" solutions have.5. Investors care about how the company plans to make them money and the depth of the technology only matters to make sure the dream founders are selling actually works (due diligence). 6. Bringing on professional investors is more about a strong collaboration between the company and its investors beyond just capital, and demonstrating a clear understanding of that aspect is crucial in convincing investors to provide funding.7. Deep tech founders are in love with their technology, but as they mature as business people, their pitch evolves to emphasize the business case and value proposition.8. It is rare for deep tech founders to talk about things they don't understand deeply with confidence. Therefore learning the business language, financials, and sales is crucial in sounding confident and credible when talking to investors.9. There are many ways to self-educate on business topics: accelerators, advisors, courses, founder communities, mastermind groups, ... but ultimately all the above has to enable the founder to validate their business and tell an exciting story about their future growth.10. Accelerators are very good ways for deep tech founders to get started, but they should educate themselves about what they need out of that experience and proactively pursue it.11. It is important to focus on building your business and securing capital, not just mentorship. It is okay to drop out of an accelerator program if it's not a good fit for your company, and you need to make sure terms you are committing to are suitable.12. When approaching a seed round, have a well-defined approach and plan. This includes having business momentum, a well-structured sales funnel, and a systematic process for dealing with investors. Align your approach with the investor's systems and workflows to ensure success in raising funds.
Data Structure for Knowledge = Language Models + Structured Data - DRT S2E13 ML Explained - Aggregate Intellect - AI.SCIENCE 2023-01-31 | 00:16 Get to know Dr. Joel Chan04:43 Joel's PhD in Cognitive Science and Transition to Human Computer Interface07:54 Exploring a patent search engine09:52 The main problem statement: how to organize artifacts to accelerate creativity 12:05 Practitioners often look for artifacts related to similar tasks to their use case12:48 Transition from knowledge repository model to expertise sharing and collaborative work17:14 Sharing artifacts with coworker without enough contextual knowledge (eg. tacit) is not helpful18:10 Area of focus; data models for collaborative knowledge work: human expertise + externalized artifacts + unstructure data24:27 Mining data on how people work is one way to create systems for knowledge sharing25:05 Slack as an example of quick recontexualization of knowledge: human interactions enriched with resources and context29:40 Creating systems that forces people into certain behavior templates doesn't work (unless it's for tasks that are very low variance)32:31 Designing the perfect externalized knowledge artifact and process really depends on the context of how people work and what they find useful 33:08 How would you get around rigidity of "expert systems" - you only need enough structure to provide value even if the system is a bit "scruffy"34:36 System requirements for an ideal collaborative knowledge work system - reusability & recontexuality, Low effort maintenance, Just in time availability + factuaity38:32 Factuality and large language models for products in highly technical areas44:54 Wrap up and verdictsKEY TAKEAWAYS:1. After research in Cognitive Science in grad school, Joel worked on creating a patent search engine that utilized text analysis and text mining systems. The goal was to add structure to unstructured full text and find interesting analogies in the patent databases.2. His focus then shifted to broader thinking about data structures, with the importance of understanding not just the problem and solution, but also the situations, constraints, and phenomena involved.3. The problem of adding structure and organization to collections of people, artifacts, resources, and patterns is important in order to enable efficient collaborative work.4. People have experimented with both automated text mining and human annotation crowdsourcing to create "expert systems" with limited satisfactory results.5. Externalizing knowledge completely and objectively is unlikely; so the best use case for knowledge artifacts is to facilitate a negotiation between tacit knowledge of collaborators and externalized artifacts, leading to the curation of knowledge.6. Slack is a tool used by developers every day to communicate with each other. It is a great example of a multiplayer HCI where "expertise sharing" (asking questions) & "knowledge sharing" (via artifacts) is deeply integrated7. Recontextualization & reuse: The ability to easily recontextualize information that was previously de-contexualized when documented is a key factor in the design of a knowledge system. It is important to be able to pick up the information and adapt it to a new use case.8. Maintenance and availability: It is important to have a system that can organize itself over time without much effort from the users / operators. It should also be available for just in time insight.9. Factuality: Being factually grounded is an important factor when designing a system that helps users navigate technical knowledge. It is essential to be able to judge the extent to which a claim is true and have a system that can help verify through provenance and evidence. 10. The addition of large language models to the system design is an interesting development, but it comes at the cost of hallucination (at least in short term and in absence of integration with reliable information sources).11. There are design questions on how to translate latent structures into machine usable structures that can be fed into information retrieval algorithms, which is a major focus in the field. However, ultimately the important factor is what workflow feels intuitive to knowledge workers and what they find useful12. The most reliable way to deisgn such system is "integrated crowdsourcing" where knowledge work structure is "mined from observing people" who are already motivated to do the work (social, process, and artifact use aspects).
EVE - Explainable Vector Embeddings - DRT S2E12 ML Explained - Aggregate Intellect - AI.SCIENCE 2023-01-25 | 00:53 Intro to the topic of explainable vector embeddings2:30 GDPR as the initial motivation to work on explainable embeddings3:26 How do you introduce semantics into decision making4:40 How can strcutured knowledge (eg. taxonomies) interact with free text in analysis7:00 Right to explanation, GDPR, ML, and revival of the phd work10:41 Going beyond just a score for semantic relatedness without incuring huge computational cost14:20 Semantic relatedness is not enough for explanation - the mode of relationship is also important (eg. can be inferred from links among wikipedia pages)15:55 How does EVE work? TrustRank (sparse version of pagerank)18:43 What is a concrete example of using EVE?21:09 Three NLP tasks that the EVE paper examines (discrimination, clustering, ranking)23:40 How are the embeddings (EVE) constructed? (comparison to Word2Vec)29:20 EVE's performance; and where would you use strcutured data in conjunction with free text in language modelling?34:29 How would EVE interact with newer models like Transformers?39:03 EVE naturally works with graphical data, but would it apply to tabular data?42:04 Amir's mandatory rant43:55 Wrap up and verdictsKey Takeaways:1. The original problem being addressed was bringing semantics to decision-making, with the use of semantic web concepts and techniques to add context to unstructured text analysis in classification tasks.2. The motivation for this approach came from the background in information retrieval and the desire to apply semantic web ideas to large volumes of free text data.3. The development of knowledge in building semantic models during the PhD led to the application of this knowledge in addressing the right to explanation requirements of the GDPR in relation to Wikipedia.4. The initial hypothesis was to use relatedness score to tackle a problem related to association between concepts, but this hypothesis did not pan out.5. Introducing explainablity to semantic relatedness exactly requires managing the memory required to retain the entire graph structure of Wikipedia.6. The main issue with the equation Atif had proposed during their PhD was that it did not explain why two concepts were related.7. EVE uses a pre-determined data structure like Wikipedia graph to introduce explainabilty by leveraging sparse vectors reperesenting all possible entity categories8. To determine the applicability of the new approach, three tasks were chosen to evaluate explainability, including discrimination test for topic modeling, clustering for grouping related items, and information retrieval for sorting out results related to a specific query.9. It was discovered that even though the model may not always be accurate, the ability for the machine to explain its reasoning can lead to better improvement strategies 10. The use of embeddings in certain tasks can outperform other methods through the explanation provided.11. Specifically, the use of explainable AI (XAI) in the context of information retrieval can be beneficial in identifying where the model is failing and fine-tuning large language models to tackle edge cases better.12. The trend of over-parameterized models with a large number of parameters, using large amounts of data and computation, and ignoring the need for structure in input data, raises concerns about the usability of these models for average users and the assumption that the model will figure out the structure on its own.
LabDAO - Decentralized Marketplace for Research in Life Sciences - DRT S2E11 ML Explained - Aggregate Intellect - AI.SCIENCE 2023-01-17 | - https://labdao.xyz/3:16 How Niklas started thinking about mechanics of research during grad school after winning a grant5:29 What was the grant money used for? and importantly what it inspired?6:56 Opportunities to do research in non-academic-checklist ways as motivation to leave academia10:55 Moving from academia to decentralized science14:01 How do scientists interact with LabDAO?15:39 What's the right model to have for a self-sustaining decentrailzed research community?21:09 Long term commitment is an important aspect of research. how's that enforced / encouraged in LabDAO?23:36 What platform is LabDAO building?25:48 What is the business model for LabDAO?29:29 What prevents the ecosystem from falling into volatile economic mechanisms we see for crypto-based systems? 33:53 How does the ecosystem handle dealing with external / larger / more traditional sources of funding?38:47 What are the traction numbers and some of the success stories? 40:53 What is the biggest challenge LabDAO dealing with?43:35 Amir's mandatory monologue!44:21 Wrap up and verdictsTakeaways:- The structure of work in academia is predetermined and competitive, with a focus on publishing papers and obtaining funding.- There was a realization that a small amount of money could be used for highly collaborative research outside of academia's traditional "playbook."- The realization sparked a change in perspective and eventually led to the creation of a startup, LabDAO, focused on creating a productive scientific environment via a decentralized research community- The problem of retention in community projects is a significant issue, as people may not be motivated to stick to a project beyond the initial excitement phase.- Segmentation, or grouping people based on shared past experiences and project ideas, can help sustain activity across a group over time.- Introducing financial incentives or governance tokens too early can obfuscate intrinsic motivation and lead to an unstable team, so it's important to give the initial clustering and momentum time to develop before introducing additional resources and mechanisms.- The community is building a platform for capturing interactions and conversations, knowledge sharing, and project building within the bioinformatics field.- The platform, called Lab Exchange, aims to provide access to compute and pre-wrapped pipelines for running analyses and machine learning models.- The platform will be based on blockchain technology and will use a peer-to-peer system for hosting and consuming services, with payments made through an open-source payment system using cryptocurrency.- There is potential for a non-profit organization to support decentralized research communities by providing infrastructure and connecting them with funders who can mature their projects further.- The idea is that traditional funding can be difficult to obtain, so a decentralized, collaborative approach can help scientists work on their ideas until they mamature, but then traditional models may still be necessary.- There are funding opportunities, such as Vita Dao, which looks for translational signs and has the capability to fund six-figure projects related to chronic diseases.
Data-Driven Behavior Change and Personalization - DRT S2E10 ML Explained - Aggregate Intellect - AI.SCIENCE 2023-01-10 | 6. 2:24 Get to know Amy3:46 Where did Amy start10:23 Digital barriers and personalization15:23 Summary of issues with personalization (prior to data driven approaches)17:10 Is behavior change hard?19:29 How do you go about changing behavior when there are barriers and motivation is not that strong24:09 What types of intervention are possible to increase the likelihood of behavior change?27:02 We need to do work on behavioral science but also on AI; which one comes first and how do they merge?30:14 Recap about transfer learning; How do you deal with organizational silos when several technical teams are involved?35:46 What is the right team composition for a product like this?38:06 How can behavior scientists and data scientists find a common language in collaboration?41:15 How do you ensure the behavior changes you introduce are ethical?44:04 Amir's long rant about privacy tech and such46:54 Verdicts and takeawaysKey Takeaways:1. The typical approach used to be long surveys on people's motivations and tasks to build heuristic algorithms.2. Constraints at the time included digital literacy, health literacy, scalability, ability to capture context, and ability to capture drift in behavior.3. Personalization technology in the past couldn't accommodate for people changing over time.4. Behavior change is hard and can depend on factors such as one's strengths, preferences, and context, level of interest in the behavior change.5. Behavior change is harder for some people due to personal barriers and lack of resources.6. Some people may struggle with behavior change, even if they understand the science behind it.7. Health behavior changes, such as weight management and weight loss, can be difficult for some people due to personal experiences, such as a history of using food as a form of love.8. A very detailed study of the issues by behavior scientists is usually the best starting point, but then there should be very close collaboration between them and AI devs9. The collaboration would require establishing a common language and operating principles and lots of iterative experiments 10. There are many types of nudges possible, but the best approach would focus on just in time intervention focused on short term gains vs long term objectives
ChatGPT - the Chatbot that Follows Instructions - DRT S2E9 ML Explained - Aggregate Intellect - AI.SCIENCE 2022-12-20 | In this episode we talked about:- how things changed since 2017 with transformers; BERT based models as encoders, and GPT based models as decoders - how scaling laws for LMs got to a point where emergent mult-tasking behavior appeared beyond certain model capacities - how prompt engineering appeared as a field to control the behavior of LLMs and problems associated with it- how instruction finetuning has lead into a promising solution to the problems associated with prompt engineering- how OpenAI team made interesting product decisions to release as a chat bot and the impact of that on creating hype- opportunities founders and other builders have to create vertical GPT based productsSummary: I think it's incredible that they've been able to get the attention of people outside of the machine learning field, and I commend the product team for their insight. However, I want to add a word of caution to the hype around the model. I've found that it still makes basic language model errors. For example, when I asked it about the smallest congressional district in Canada, it gave me the incorrect answer. I think it's important to remember that while this model is a significant technical progress, there are still many shortcomings and it's crucial to think about the problem we're trying to solve and the right system design for achieving that goal when working with these models.
Investing in Deep Tech - Investors Angle; Deep Random Talks S2E8 - Ft. Moien Giashi, Amir Feizpour ML Explained - Aggregate Intellect - AI.SCIENCE 2022-12-16 | In this episode Moien and I talk about his journey into startup investing, and things he has learned along the way.
Modern Knowledge Management in 2022 - Deep Random Talks S2E7 ML Explained - Aggregate Intellect - AI.SCIENCE 2022-12-09 | What frustrates you the most with personal knowledge management? Participate in this week's Deep Random Talks and learn about modern knowledge management from the experts- Amir Feizpour, Ammar Khan and Stian Håklev.
TalentDAO- How does decentralized scientific publishing work - Deep Random Talks S2E6 ML Explained - Aggregate Intellect - AI.SCIENCE 2022-12-02 | Listen to Amir Feizpour, Carla Ostmann and Renee Davis speak about how decentralized scientific publishing work.
Evaluating Performance of Large Language Models with Linguistics - Deep Random Talks S2E5 ML Explained - Aggregate Intellect - AI.SCIENCE 2022-11-25 | Listen to Amir Feizpour, Serena McDonnell and Bai Li speak about Natural Language Processing on DRT's this episode.
Second Brain for Technical Knowledge Management- DRT S2 E4 ML Explained - Aggregate Intellect - AI.SCIENCE 2022-11-18 | In this episode of Deep Random Talk, our host, Amir Feizpour, and co-host, Ammar Khan, speak to Ozan Ozyegen regarding second brain and how it helped him in his studies and work.
What are the future plans of Foodshake? ML Explained - Aggregate Intellect - AI.SCIENCE 2022-11-14 | ...
Learn about Foodshake and it’s vegan recipes! ML Explained - Aggregate Intellect - AI.SCIENCE 2022-11-14 | ...
Community-Driven Product Development - Deep Random Talk S2E3 ML Explained - Aggregate Intellect - AI.SCIENCE 2022-11-10 | Catch Amir Feizpour and Roxana-Maria Barbu speak with Serena McDonnell about her startup and how she wore multiple hats while working full-time.
Funding Deep Tech Projects: Founders POV - Deep Random Talk S2E2 - Ft. Moien Giashi, Amir Feizpour ML Explained - Aggregate Intellect - AI.SCIENCE 2022-11-03 | invest in aggregate intellect: https://ai.science/investnotes and resources: https://ai.science/l/6e39071c-9575-45aa-9536-dc87f7c26f22@/assets
Jiri Stodulka- Data Scientist and Investor of Aggregate Intellect ML Explained - Aggregate Intellect - AI.SCIENCE 2022-10-31 | Jiri Stodulka has been a part of the Aggregate Intellect community since 2019. Learn why he chose to invest and support us! #shorts #aggregateintellect #iownAI
Private Investing in Deep Tech - DRT - S2 bonus ep - Ft. Moien Giashi, Darryl Kirsch, Hessie Jones ML Explained - Aggregate Intellect - AI.SCIENCE 2022-10-26 | invest in aggregate intellect: https://ai.science/investSpeakers:linkedin.com/in/amirfzprlinkedin.com/in/moien-giashilinkedin.com/in/darryl-kirsh-7405a21blinkedin.com/in/hessiejones1In this session we will be talking to some startup and investment veterans about questions like* why they are interested in investing in deep tech* what the differences between "private investing" and other types of investment are* what major investment opportunities they see coming up in deep tech* what major investment opportunities they see coming up in AI* what advice they have for those who want to start investing in startups
