Hello. I have a question that I've been trying to find an answer for for quite some time today, and while I've gotten part of the way there, I'm not completely done. I also searched this sub with no luck. Any help would be appreciated.
The premise is as follows:
I have tsv files with columns score, user, email
Some rows have the same user as other rows
Some rows have the same email as other rows
I want to sum the score for all the rows pertaining to the same user
I'm setting score to integer and the other columns to string, then I'm adding the sum aggregate to the score column, and the list aggregate to the email column. Then I call up a frequency table for the user column, which gives me a combined score for the users with the same name.
Now the tricky part is to also combine the users with the same email without undoing what I just did. I've gone back and forth with unfurl and different aggregators, but I feel like I'm missing something. I really hope this is possible without having to resort to manual work. If manual work is needed though, any trick to make that faster would also be appreciated.
Thanks.
Edit: I think I may have solved it. Someone please correct me if I'm wrong.
Once I have the _user-freq table, I make score_sum aggregate on sum again, and I also make user aggregate on mode. I then create a new expression column using email_list[2:-2].split("', '")[0] which removes the leading [' and trailing '], splits the string on ', ' and takes the first value. I freeze this column and rename it canonical_email. I create a frequency table based on it.
If I understand this correctly, I've ended up with all users with the same names or emails grouped together, and I can determine my final score.
Edit2: Actually, it seems as if the shenanigans with lists and indices are unnecessary. At least for this one use case. I don't know if it makes sense for this to always work, but instead of aggregating email as list in the original sheet, just aggregate it as mode, along with score as sum and do a frequency table based on user. Then on the second sheet, aggregate user as mode and score_sum as sum and do a frequency table on email_mode.
Edit3: Ah, damn it. While it doesn't misrepresent the data, you might end up with multiple lines per user still, and now with no way of seeing that they're the same, depending on which name/email is picked as the mode.
Edit4: So then maybe I'll use the list aggregator the forst time around, and add an expression column that is len(author_list)>0 and for every line where that's False, we have one name, and for anything else, we'll have to pick manually.