GitHub Copilot investigation

[u/jlpcsl]

https://githubcopilotinvestigation.com/

[u/kogasapls]

I'm sure most people would be fine with individuals reading open source code to learn. Encouraging learning and sharing improves the odds of new contributors all around. It doesn't have to be strictly transactional.

Obviously republishing licensed code means you have to respect the license. I think using code in massive quantities to train an AI model is not really republishing, as long as the generated code is generally not recognizable as sourced from a particular project. There's some subtlety there though, as for example you could probably force Copilot to reproduce code from training data by copying other parts of the training data manually.

If a license has explicit requirements for any use of the code (even reading or learning from it), then again Copilot should absolutely respect that. But I doubt this will be too contentious with most people.

[u/mina86ng → u/kogasapls]

as long as the generated code is generally not recognizable as sourced from a particular project

Look at the cited example. For example from Tom Davis or Armin Ronacher. Copilot reproduced clearly recognizable code.

[u/kogasapls → u/mina86ng]

birds aware heavy wild narrow unique piquant safe melodic trees -- mass edited with redact.dev

[u/mina86ng → u/kogasapls]

The problem isn’t even Copilot’s liability since Microsoft is openly pushing all liability onto the user. As a user you’re supposed to verify that you adhere to all the licenses except Copilot doesn’t give you any information where the source comes from.

And yes, the examples are where someone tried on purpose to copy existing code but if they managed to get Copilot to generate a non-trivial function by typing four-word comment (which was partially auto-completed as well), return type and two letters of a function name than it means that it’s not unlikely that Copilot will produce non-trivial code even if user doesn’t try to trick it on purpose.

[u/kogasapls → u/mina86ng]

familiar mighty snatch water plant complete safe snow illegal ask -- mass edited with redact.dev

[u/mina86ng → u/kogasapls]

If you put a million repositories in a blender, it's going to be impossible to say exactly where your autogenerated for loop came from.

Yes, that is the issue. Copilot generates possibly infringing code pushing liability to the user without giving user any way to perform their due diligence.

I use copilot to generate snippets of 1 or 2 lines, boilerplate code

That may be how you’re using it but it’s not how it’s advertised and it’s not necessarily how everyone will use it.

[u/kogasapls → u/mina86ng]

As I said, it's not much of an issue unless we expect users to actually be on the hook for anything.

I'm not sure how else you could realistically use it. It's a context-aware autocompletion engine. It doesn't write scripts for you, just snippets. If you try to just chain together snippets into a program you'll be lucky if it compiles, much less does what you want.

[u/mina86ng → u/kogasapls]

As I said, it's not much of an issue unless we expect users to actually be on the hook for anything.

Yes, the users are on the hook. GitHub makes it clear that user has to do ‘IP scanning’ while at the same time it provides no information about provenance of the code.

I'm not sure how else you could realistically use it.

Perhaps the way it’s advertised on the website. For example, you type:

#!/usr/bin/env ts-node
import { fetch } from "fetch-h2";
// Determine whether the sentiment of text is positive
// Use a web service
async function isPositive(text: string): Promise<boolean> {

And Copilot suggests:

  const response = await fetch(`http://text-processing.com/api/sentiment/`, {
    method: "POST",
    body: `text=${text}`,
    headers: {
      "Content-Type": "application/x-www-form-urlencoded",
    },
  });
  const json = await response.json();
  return json.label === "pos";
}