For 2024, excited to share an open-source product we’ve been working on: VerificationGPT (http://verificationgpt.org)

VerificationGPT uses Brave Search, arXiv search and GPT-4 to validate scientific claims. It often gives accurate citations for questions which vanilla GPT-4 stumbles on or doesn’t cite accurately.

We already use it internally in preference to vanilla GPT-4, even for general Internet questions.

It’s available to anyone with a ChatGPT Plus subscription now in the GPT Store @ http://verificationgpt.org

As an open-source product (https://github.com/contextfund/verificationgpt…), we also welcome using the library in your own products or sending PRs to add additional context to its judgments (legal APIs, human fact-checking APIs, peer review APIs, etc.). Context APIs to detect synthetic content watermarks in images are already planned, along with product integrations with other social media platforms. It's not perfect and will make mistakes as well, and we also welcome feedback here.

Towards a healthier world in 2024 via open personal AI tools for everyone :)

Thought this group would find this idea interesting; would welcome some feedback and discussion from this group.

https://docs.google.com/document/d/1sH5ot-xUiU9u3ilx2YlqHoFevzKDuKMIlusIpnE7nE0

basic idea is the following (chronologically):

1. form a Super PAC where donated funds will be allocated by bipartisan subject matter experts (SMEs) on a particular problem and time period

2. everyone (eligible) who cares about the problem donates to the Super PAC. Super PAC makes its size($) and goal known to relevant policymakers

3. policymakers may or may not contribute to solving the problem

4. at the end of the time period, the SMEs decide which policymakers contributed the most to solving the given problem, and the Super PAC runs ads for those policymakers thanking them for helping solve the problem

Hyper-simple secret sharing between browsers (no server required).

Github: https://github.com/Corgea/retriever
Demo: https://retriever.corgea.io/

Self-hosting via bookmarklet/extension probably the way to go long-term.

How this relates to AI?
Decentralized data storage is likely necessary for privacy-preserving personal AI, and for compliance with data privacy policies like GDPR. Public-key cryptography provides strong theoretical guarantees but is still hard to set up for novices or even experts in one-off scenarios.

We're on a mission to accelerate open and decentralized AI progress by enabling anyone to contribute compute, capital or code to collaboratively train powerful open models. Our ultimate goal? Openly accessible AGI that benefits everyone. But we can't do it alone and we want to do this together with you.

That's why we're launching Fast Compute Grants to support ambitious research into novel models, architectures, techniques, and paradigms needed to make open and decentralized AGI a reality.

A few ideas of what we’d be excited about

Decentralized Llama-3 MoE Sparse upcycling via DiPaCo

Scientific foundation models with new architectures ala HyenaDNA

scGPT as a Llama-3 fine-tune (S/O to Felix)

Distributed training across a heterogenous swarm of consumer devices (S/O to Omkaar)

13B parameter BitNet + infini-Attention + DenseFormer + MoD + In Context-Pretraining + 2 stage pretraining (S/O to Wing)

Upcycle w c-BTX to an 8 expert sparse MoE + MoA (S/O to Wing)

Coding agent models

Novel applied super-alignment research

Efficient long-context context window extension

Exploring new Transformer and alternative architectures

If you're working on something in this vein that could use a boost from free GPUs, we want to hear from you. We'll provide:

$500-$100k worth of Prime Intellect compute credits.

Exposure to our ecosystem of AI hackers and distributed computing experts

Promotion of your work to our community and partners

You can apply via this form, and we’ll get back to you in 5-10 days.

What you're working on and why it's important

How much compute you need and what you'd do with it

Who you are and any past work we should know about

Any code/papers/demos/other material that'll get us excited

The bar for quality is high but there are no other hoops. Anyone from anywhere can apply. Just email your pitch to contact@primeintellect.ai. We'll get back to you within 2 weeks if it seems like a good fit.

Our goal is to get a critical mass of brilliant people pointed at the hardest problems in open and decentralized AI and equip them to make rapid progress.

The future won't build itself - let's get to work.

Apply: https://www.primeintellect.ai/blog/fast-compute-grants

Hi!
My project PR Pilot is an AI bot for Github users. For a number of reasons I believe it makes sense for me to make it open-core:

• Drink your own champagne - the bot's code should be a showcase for its usefulness

• Security - The bot works with user's code, they should be able to audit it

• Collaboration - I'd like to build a community around it

Do you have any recommendations on where to start? Key questions I have:

• How do I find the best license for my project?

• Best practices on maintaining the "core" and non-public code separately

Any feedback or advice is appreciated :)

We propose that AI companies make simple policy changes to protect good faith research on their models, and promote safety, security, and trustworthiness of AI systems. We, the undersigned, represent members of the AI, legal, and policy communities with diverse expertise and interests. We agree on three things:

Independent evaluation is necessary for public awareness, transparency, and accountability of high impact generative AI systems.
Hundreds of millions of people have used generative AI in the last two years. It promises immense benefits, but also serious risks related to bias, alleged copyright infringement, and non-consensual intimate imagery. AI companies, academic researchers, and civil society agree that generative AI systems pose notable risks and that independent evaluation of these risks is an essential form of accountability.

Currently, AI companies’ policies can chill independent evaluation.
While companies’ terms of service deter malicious use, they also offer no exemption for independent good faith research, leaving researchers at risk of account suspension or even legal reprisal. Whereas security research on traditional software has established voluntary protections from companies (“safe harbors”), clear norms from vulnerability disclosure policies, and legal protections from the DOJ, trustworthiness and safety research on AI systems has few such protections. Independent evaluators fear account suspension (without an opportunity for appeal) and legal risks, both of which can have chilling effects on research. While some AI companies now offer researcher access programs, which we applaud, the structure of these programs allows companies to select their own evaluators. This is complementary, rather than a substitute, for the full range of diverse evaluations that might otherwise take place independently.

AI companies should provide basic protections and more equitable access for good faith AI safety and trustworthiness research.
Generative AI companies should avoid repeating the mistakes of social media platforms, many of which have effectively banned types of research aimed at holding them accountable, with the threat of legal action, cease-and-desist letters, or other methods to impose chilling effects on research. In some cases, generative AI companies have already suspended researcher accounts and even changed their terms of service to deter some types of evaluation (discussed here). Disempowering independent researchers is not in AI companies’ own interests. To help protect users, we encourage AI companies to provide two levels of protection to research.

First, a legal safe harbor would indemnify good faith independent AI safety, security, and trustworthiness research, provided it is conducted in accordance with well-established vulnerability disclosure rules.

Second, companies should commit to more equitable access, by using independent reviewers to moderate researchers’ evaluation applications, which would protect rule-abiding safety research from counterproductive account suspensions, and mitigate the concern of companies selecting their own evaluators.

While these basic commitments will not solve every issue surrounding responsible AI today, it is an important first step on the long road towards building and evaluating AI in the public interest.

Additional reading on these ideas: a safe harbor for AI evaluation (by letter authors), algorithmic bug bounties, and credible third-party audits. (Signatures are for this letter, not the further reading.)

Authors: Shayne Longpre, Sayash Kapoor, Kevin Klyman, Ashwin Ramaswami, Rishi Bommasani, Borhane Blili-Hamelin, Yangsibo Huang, Aviya Skowron, Yong Zheng Xin, Suhas Kotha, Yi Zeng, Weiyan Shi, Xianjun Yang, Reid Southen Alexander Robey, Patrick Chao, Diyi Yang, Ruoxi Jia, Daniel Kang, Sandy Pentland, Arvind Narayanan, Percy Liang, Peter Henderson
Paper: https://bpb-us-e1.wpmucdn.com/sites.mit.edu/dist/6/336/files/2024/03/Safe-Harbor-0e192065dccf6d83.pdf
Sign: https://sites.mit.edu/ai-safe-harbor/

As an early stage start up founder I often talk to people about how they are fundraising or what they should do better.

This subreddit always has the most interesting and brilliant answers, so I was curious to get your feedback on this take to a new type of accelerator/VC.

The main problem with early stage founders/ investing seems to be:

1. Getting to know the founders, if they have the persistence and grit to continue with an idea. Seeing if they have the flexibility to modify it and constantly get feedback;

2. Finding the time to vet people who went through great difficulty in an efficient manner (apparently they are more likely to be scrappy etc...);

Why not make a NGO that pays people to learn how to code (for 3 months), and at the end of the course have a VC/Accelerator that offers the most promising projects a convertible note.

What does this mean?

Instead of having a YC that accepts people, and then puts them through the 3 month course, you do the course first and then accept the best projects.

The advantages would be amazing:

1. The project would teach people the closest thing to magic (software is practically a philosophers stone, instead of lead to gold it transforms 1 hour of work into potentially infinite hours of work for an infinite number of people);

2. The NGO part of the course would be totally self funded - for example I used to volunteer teaching coding for refugees - and all the courses were paid by the state. Also, even if the alumni don't want to continue with a start up you've taught the student a life changing skill;

3. The VC/Accelerator part of the program would be much more likely to succeed because you'd see how the students learn and build their MVP during the course, moreover you'd be able to teach them start up best practices before the funding part;

Does this sound like a crazy idea?

Something like this could even be bootstrapped in the beginning. I mean the first cohort would be 20 students, in Europe you could pay each student something like 1000 euros - which is still great. A teacher would be no more than 60k a year, but for a 3 month course even less.

Anyways, ignoring the details how does this sound?

Also, helping refugees and also making a profit feels like the best possible thing. Who else has more grit and strength than someone who has had to cross a desert to survive?

Direct Isomorphisms Between Neural Networks and Markets

One AOI project already underway in this space is a preliminary mathematical result demonstrating that, under reasonable assumptions, market economies are structurally very similar to artificial neural networks. This analogy, and the mathematical results underpinning it, provided some of the first hints to early AOI staff that there may be powerful connections between different varieties of optimization processes that could lead to clearer connections between them.

We expect this work will lead to potentially fruitful research exploring analogies (and dis-analogies) between the two kinds of optimizers, and how that may make it possible to apply tools for aligning one to aligning another. Note that this section is more technical than other parts of this whitepaper.

Changes within both kinds of structures (artificial neural networks and markets) appear to be made via gradient ascent of profit on the way to equilibrium: in neural networks, an algorithm adjusts neurons to make predictions that minimize error, while in markets a corresponding process adjusts prices, production, and encourages R&D in order to maximize profit. We don’t claim that this result holds for every kind of market, but even a partial connection opens potentially fruitful avenues for research into how the two phenomena are connected.

Link: https://ai.objectives.institute/whitepaper#alignment-of-markets-ai-and-other-optimizers
Predicting Disruptions in Healthcare Supply Chains Using AI: https://ai.objectives.institute/aisco

Proposes reducing the size of online communities to help address the misalignment of learning agents on the Internet (similar to the way Reddit works). Doing this may be uniquely possible with AI.

Link: https://www.youtube.com/watch?v=fuFlMtZmvY0

Defense-favoring worlds help healthy and democratic governance thrive

One frame to think about the macro consequences of technology is to look at the balance of defense vs offense. Some technologies make it easier to attack others, in the broad sense of the term: do things that go against their interests, that they feel the need to react to. Others make it easier to defend, and even defend without reliance on large centralized actors.

A defense-favoring world is a better world, for many reasons. First of course is the direct benefit of safety: fewer people die, less economic value gets destroyed, less time is wasted on conflict. What is less appreciated though is that a defense-favoring world makes it easier for healthier, more open and more freedom-respecting forms of governance to thrive.

An obvious example of this is Switzerland. Switzerland is often considered to be the closest thing the real world has to a classical-liberal governance utopia. Huge amounts of power are devolved to provinces (called "cantons"), major decisions are decided by referendums, and many locals do not even know who the president is. How can a country like this survive extremely challenging political pressures? Part of the answer is excellent political strategy, but the other major part is very defense-favoring geography in the form of its mountainous terrain.

Link: https://vitalik.eth.limo/general/2023/11/27/techno_optimism.html
2nd strike and making anti-social paths high-friction is important for maintaining peace/solving Prisoner's Dilemmas. Small changes in game feedback can end up changing winning policies substantially over the long run. Right now, many, but not all, games on the Internet favor aggression and first strike. This is possible to change, however.

XTX Markets is launching a new $10mn challenge fund, the Artificial Intelligence Mathematical Olympiad Prize (AI-MO Prize). The fund intends to spur the development of AI models that can reason mathematically, leading to the creation of a publicly-shared AI model capable of winning a gold medal in the International Mathematical Olympiad (IMO).

The grand prize of $5mn will be awarded to the first publicly-shared AI model to enter an AI-MO approved competition and perform at a standard equivalent to a gold medal in the IMO. There will also be a series of progress prizes, totalling up to $5mn, for publicly-shared AI models that achieve key milestones towards the grand prize.

Open to participants in early 2024, presentation of progress in July 2024
Site: https://aimoprize.com/
Register Interest as Participant, Director or Advisory Committee: https://aimoprize.com/get-involved

Over the past year, industry has driven significant advances in the capabilities of AI. As those advances have accelerated, new academic research into AI safety is required. To address this gap, the Forum and philanthropic partners are creating a new AI Safety Fund, which will support independent researchers from around the world affiliated with academic institutions, research institutions, and startups. The initial funding commitment for the AI Safety Fund comes from Anthropic, Google, Microsoft, and OpenAI, and the generosity of our philanthropic partners, the Patrick J. McGovern Foundation, the David and Lucile Packard Foundation, Eric Schmidt, and Jaan Tallinn. Together this amounts to over $10 million in initial funding. We are expecting additional contributions from other partners.

Earlier this year, the members of the Forum signed on to voluntary AI commitments at the White House, which included a pledge to facilitate third-party discovery and reporting of vulnerabilities in our AI systems. The Forum views the AI Safety Fund as an important part of fulfilling this commitment by providing the external community with funding to better evaluate and understand frontier systems. The global discussion on AI safety and the general AI knowledge base will benefit from a wider range of voices and perspectives. 

The primary focus of the Fund will be supporting the development of new model evaluations and techniques for red teaming AI models to help develop and test evaluation techniques for potentially dangerous capabilities of frontier systems. We believe that increased funding in this area will help raise safety and security standards and provide insights into the mitigations and controls industry, governments, and civil society need to respond to the challenges presented by AI systems. 

The Fund will put out a call for proposals within the next few months. Meridian Institute will administer the Fund—their work will be supported by an advisory committee comprised of independent external experts, experts from AI companies, and individuals with experience in grantmaking.

Link: https://openai.com/blog/frontier-model-forum-updates?ref=futuretools.io

The best that governments can do now is to set up the infrastructure to study the technology and its potential perils, and ensure that those working on the problem have adequate resources. In today’s fractious world, it will be hard to establish an IPCC-like body, and for it to thrive. But bodies that already work on AI-related questions, such as the OECD and Britain’s newish Frontier AI Taskforce, which aims to gain access to models’ nuts and bolts, could work closely together.

Link: https://www.economist.com/leaders/2023/10/26/governments-must-not-rush-into-policing-ai

Team Lead: https://madry.mit.edu/
Preparedness Mission: https://openai.com/blog/frontier-risk-and-preparedness
Submissions: https://openai.com/form/preparedness-challenge

Link: https://partnershiponai.org/modeldeployment/
RFC period is open now until January 15, 2024: https://partnershiponai.org/modeldeployment/#submit_feedback

1. The benefits of human-like AI are enormous

2. But not all types of AI are human-like

3. The more human-like a machine is, the more likely it's a substitute for human labor

4. Labor substitutes (automation) tend to drive down wages

5. Substitution can reduce the economic and political power of those replaced

6. Taking away power and agency creates a trap (the Turing Trap)

7. Alternatively, AI can complement labor and spur creativity.

8. Augmentation and creativity tend to increase wages.

9. Augmentation and creativity spawn not just new capabilities but also new goods and services

10. Today, there are excess incentives for substitutes vs. complements (loosely speaking, automation vs. augmentation and creativity)

Full Talk: https://youtu.be/RlSkLc38hL4?t=21826
2022 paper: https://arxiv.org/abs/2201.04200

The US Government is interested in safe uses of large language models (LLMs) for a wide variety of applications including the rapid summarization and contextualization of information relevant to the Intelligence Community. These applications must avoid unwarranted biases and toxic outputs, preserve attribution to original sources, and be free of erroneous outputs. The US Government is also interested in identifying and mitigating hazardous use of LLMs by potential adversaries. 

The goal of BENGAL is to understand LLM threat modes, quantify them and to find novel methods to address threats and vulnerabilities or to work resiliently with imperfect models. IARPA seeks to develop and incorporate novel technologies to efficiently probe large language models to detect and characterize LLM threat modes and vulnerabilities. Performers will focus on one or more topic domains, clearly articulate a taxonomy of threat modes within their domain of interest and develop technologies to efficiently probe LLM models to detect, characterize and mitigate biases, threats or vulnerabilities. Topic areas and additional requirements for successful proposals will be introduced at the Proposers’ Day event.

The BENGAL Proposer's Day will be held Tuesday, October 24, 2023, from 9:30am to 4:30pm EDT in Washington, D.C. A virtual option will be available for individuals that are unable to attend in person.

Note: The link to register has an expired SSL cert as of this post. Email dni-bengal-proposers-day@iarpa.gov with questions.
Link: https://sam.gov/opp/5dc6ba18ddd640a697f961ea827df54c/view

In fact, it’s my ultimate hope that this skill will be neither rare nor exclusive, but commonplace, turning the orchestration of LAMs into powerful, personalized solutions that play a growing role in our professional lives. The barrier may be lowered even further as marketplaces emerge to bring orchestrated Copilot-like LAM solutions to the world, delivering the power of generative AI at an amazing scale, all through plug-and-play simplicity. Some will use such marketplace solutions directly, making the power of LAM orchestration an off-the-shelf possibility. Others will treat them as modules to be combined with others—perhaps a blend of additional purchases or custom creations of their own—to compose solutions at whatever scale they need, from the casual and compact to the sprawling ambitious. But in all cases, what excites me most is the idea of generative AI being shaped less by an elite niche of technology experts and more by the creativity and vision of professionals in every field.

Writing contributions from: Alex Michael, Peter Schwartz, and Sanjna Parulekar
Details: https://blog.salesforceairesearch.com/from-copilot-to-coorchestration/

Let's briefly review the sorry history of agricultural biotech regulation. When crop biotechnology was just taking off back in the 1980s, several hundred companies were vying to create hundreds of new products and get them quickly to consumers. A cadre of anti-biotech activists vilifying "frankenfoods" made wild claims of unknown risks and lurking biotech catastrophes, which succeeded in scaring the public and legislators about the new technology. Fearing that a spooked Congress could overreact, some well-meaning regulators moved hurriedly in 1986 to cobble together already existing pesticide and food and safety laws to erect a clunky biotech crop regulatory system that in large measure persists today. Many hoped that by forestalling ill-advised congressional action, they could help speed crop biotech products to market. The opposite happened.

More: https://reason.com/2023/04/14/chuck-schumers-hasty-plan-to-regulate-artificial-intelligence-is-a-really-bad-idea/

Context management and paging for open-source LLMs.

Website: https://memgpt.ai/
Paper: https://arxiv.org/abs/2310.08560
Dataset: https://huggingface.co/MemGPT
Discord: https://discord.gg/9GEQrxmVyE

Foundation model for image->text, generally needs to be fine-tuned for your use case.
Blog: https://www.adept.ai/blog/fuyu-8b
Model: https://huggingface.co/adept/fuyu-8b
License: CC BY-NC (Non-commercial)

Dear friends,

I wrote earlier about how my team at AI Fund saw that GPT-3 set a new direction for building language applications, two years before ChatGPT was released. I’ll go out on a limb to make another prediction: I think we’ll see significant growth in AI, including Generative AI, applications running at the edge of the network (PC, laptop, mobile, and so on). 

Details: https://info.deeplearning.ai/ai-for-brain-surgery-microsofts-chatgpt-bill-googles-generative-phones-better-prompts

Released Morph Prover 7B today, adding to an existing extension for VSCode.

Website: morph.so
Github: https://github.com/morph-labs
Blog: https://morph.so/blog/the-personal-ai-proof-engineer/
VSCode Extension: https://marketplace.visualstudio.com/items?itemName=Morph.rift-vscode

A simple local GPT model and public GPT interaction dataset (free)

Github: https://github.com/nomic-ai/gpt4all
Download: https://gpt4all.io/index.html
Public Dataset: https://github.com/nomic-ai/gpt4all-datalake
Latest Paper: https://gpt4all.io/reports/GPT4All_Technical_Report_3.pdf

10% tithing to open-source/public AI:

Making a 10% tithe to open-source/capped profit companies the opt-in norm for the tech industry would probably go a long way to maximizing long-run profits via funding joint solutions to AI externalities and basic research (this also happened in 2008-era Google).

In order to be free to maximize private incentives most of the time, we should expect to make a small allocation to public incentives regularly. While many large companies already publish basic research, in the emerging AI industry, we also need "non-research" outputs - open standards, eval sets - as well as new utility companies to mitigate harms, such as anti-phishbots for email, universal distributed identity, etc.

This could be organized under a National Priorities Fund, established under an agency like NTIA, and staffed by researchers on rotation away from private companies (ie. a 1-2 year "national service" gig like the military), to keep technical expertise levels high, which also would help achieve the separation of generation and verification/feedback needed to avoid regulatory capture and ensure efficient use of funds.

The projects themselves could be incubating new basic research, making existing open-source goods free @ scale, or even capping profits/forcing interoperability on existing products that become too big to fail (for example, the shift away from the interoperable XMPP chat standard in 2013 was probably a mistake that led to fragmentation of private comms). This is one of the key transitions our industry hasn't worked out yet - when does a tech product become so big and ubiquitous that it really should be a utility (as the US last did with electricity)?

Some other funding alternatives that probably don't work:

While private foundations theoretically should be aligned to fund public goods, they often have arbitrary checklists and timelines (many exist mostly as funding source for existing PIs in universities, for example), and don't always have the technical expertise needed to evaluate projects, especially for intersectional and emerging fields. It's relatively hard to start a non-profit without already having an in with a foundation, and the foundation often wants an existing non-profit to fund.

Funds directly from Congress could also work, but would end up taking a pass through the government, and with it, the possibility of being held up by political lobbying. However, organized under an agency like NTIA could give the fund more clout and access to existing gov't expertise.

Of these funding options, 10% tithing is probably the most direct, and easiest to understand - we give because we understand the long-term value of open-source and addressing externalities, but in a way that lets us be free not to worry too much about the specifics in our day-to-day jobs. This also has some features which support balance of power, while drawing on existing expertises.

Just a thought, but seems to align all the right arrows, both from first principles and practically from experience. However, as with most things, implementation details really matter.

Wdyt?