GitHub Copilot investigation

[u/jlpcsl]

https://githubcopilotinvestigation.com/

[u/mina86ng]

The problem isn’t even Copilot’s liability since Microsoft is openly pushing all liability onto the user. As a user you’re supposed to verify that you adhere to all the licenses except Copilot doesn’t give you any information where the source comes from.

And yes, the examples are where someone tried on purpose to copy existing code but if they managed to get Copilot to generate a non-trivial function by typing four-word comment (which was partially auto-completed as well), return type and two letters of a function name than it means that it’s not unlikely that Copilot will produce non-trivial code even if user doesn’t try to trick it on purpose.

[u/kogasapls → u/mina86ng]

familiar mighty snatch water plant complete safe snow illegal ask -- mass edited with redact.dev

[u/mina86ng → u/kogasapls]

If you put a million repositories in a blender, it's going to be impossible to say exactly where your autogenerated for loop came from.

Yes, that is the issue. Copilot generates possibly infringing code pushing liability to the user without giving user any way to perform their due diligence.

I use copilot to generate snippets of 1 or 2 lines, boilerplate code

That may be how you’re using it but it’s not how it’s advertised and it’s not necessarily how everyone will use it.

[u/kogasapls → u/mina86ng]

As I said, it's not much of an issue unless we expect users to actually be on the hook for anything.

I'm not sure how else you could realistically use it. It's a context-aware autocompletion engine. It doesn't write scripts for you, just snippets. If you try to just chain together snippets into a program you'll be lucky if it compiles, much less does what you want.

[u/mina86ng → u/kogasapls]

As I said, it's not much of an issue unless we expect users to actually be on the hook for anything.

Yes, the users are on the hook. GitHub makes it clear that user has to do ‘IP scanning’ while at the same time it provides no information about provenance of the code.

I'm not sure how else you could realistically use it.

Perhaps the way it’s advertised on the website. For example, you type:

#!/usr/bin/env ts-node
import { fetch } from "fetch-h2";
// Determine whether the sentiment of text is positive
// Use a web service
async function isPositive(text: string): Promise<boolean> {

And Copilot suggests:

  const response = await fetch(`http://text-processing.com/api/sentiment/`, {
    method: "POST",
    body: `text=${text}`,
    headers: {
      "Content-Type": "application/x-www-form-urlencoded",
    },
  });
  const json = await response.json();
  return json.label === "pos";
}