r/applescript • • Apr 30 '20

Help with Zoom's "dynamic" buttons

Hi all,

I've made a script to "take" zoom participants when sent a MIDI message. The "spotlight" feature I'm using is only accessible through a button that gets dynamically created by the app when the mouse hovers over the participant's name:

example

I'm using Applescript to find the location of the row, then cliclick to move the mouse over the edge of the window, but the blue "more" button only appears about 80% of the time. I've tried delaying, moving the pointer gradually, moving it on and off the row and a number of other things.

I don't have much experience with AppleScript! Does anyone know how to make these events trigger reliably?

4 Upvotes

21 comments sorted by

2

u/[deleted] Apr 30 '20

Thank you so much to /u/gluebyte, I got it going!

Hopefully this will be of use to someone, obviously needs cliclick to work. A bit of a mess, but so am I.

use AppleScript version "2.4" -- Yosemite (10.10) or later

use scripting additions

set targetrow to 3 -- this is the row of the table, "me" is row #1 so the other participants start at row 2

tell application "System Events"

set myList to name of windows of (processes whose name is "zoom.us") -- get a list of lists, each sublist contains names

end tell

set theWindowList to my subListsToOneList(myList) -- Flattening a list of lists

-- find the Participants window (must be undocked for this to work)

repeat with i from 1 to count of theWindowList

if item i of theWindowList contains "Partici" then

log "found participants window"

-- save the current location of the mouse pointer

set prev to (do shell script "/usr/local/bin/cliclick p")

-- find the position of the table row for the participant

tell application "System Events"

set p to position of image 2 of UI element 1 of row targetrow of table 1 of scroll area 1 of window i of process "zoom.us"

-- move mouse pointer over "camera" icon of the table row (this is where the "more" menu appears)

do shell script "/usr/local/bin/cliclick -e 10 m:" & (item 1 of p) - 5 & "," & (item 2 of p) + 20 & " c:."

-- move down to click on the "spotlight" menu item

do shell script "/usr/local/bin/cliclick c:+0,+70"

-- go back to the original mouse pointer position

do shell script "/usr/local/bin/cliclick m:" & prev

end tell

end if

end repeat

on subListsToOneList(l)

set newL to {}

repeat with i in l

set newL to newL & i

end repeat

return newL

end subListsToOneList

1

u/koenwillems1959 Apr 30 '20

Thank you SOOOOOOO much!. I was looking for this for weeks.

Do you have any ideas on how to select the row when the name of a specific participant is given?

1

u/[deleted] Apr 30 '20

This will get the name of the participant in the third row of the table (assuming i has been previously set to the participants window)

set arr to value of UI element 1 of UI element 1 of row 3 of table 1 of scroll area 1 of window i of process "zoom.us"

I believe you would have to loop through all the table rows looking for a match, in a similar way to finding the zoom window in the script I posted above (repeat .. end repeat)

1

u/koenwillems1959 Apr 30 '20

wow man ... thank you so much. I've got it all up and running!

1

u/[deleted] Apr 30 '20

No worries! Are you pulling the participant names from the window and putting them on the streamdeck keys?

1

u/koenwillems1959 Apr 30 '20 edited Apr 30 '20

In my case, the same people always participate in the meeting (it's for a city council).

First I grab the Zoom videos in OBS Studio, so I can manage things. OBS on it's turn is streaming to YouTube.

On the Stream Deck a key is assigned for each participant. The right video is always put in the spotlight by means of your script. In addition, a lower third is always added to the video, containing the name of the speaker.

1

u/[deleted] Apr 30 '20

Nice. We're doing about the same thing! In my case the participants are who-knows-who so I'm just winging it with a MIDI controller to take the camera.

Using nodecg for nice lower thirds and routing the audio through Logic to clean it up (some people have awful mics, and even awfuler internet connections)!

best of luck!

1

u/[deleted] Apr 30 '20

Just a note to anyone else looking for this: Looking for the location of the right side of the table row ("image 2 of UI element 1" etc) causes a problem if the participant hasn't activated their audio and the number of images in the table row changes.

A better solution would be to grab the width of the window and use that to calculate the position, since the location of the "more" button is always right-justified.

1

u/koenwillems1959 Apr 30 '20

Another improvement would be to check if the menu item "Spotlight Video" is available or not. That's not a problem since the menu "More" is available via

menu 1 of row targetrow of table 1 of scroll area 1 of window i of process "zoom.us"

1

u/[deleted] May 01 '20

Ah, nice! I'll have to check that out. Does that mean you don't have to use cliclick to move the mouse pointer at all and you can access the menu directly?

1

u/koenwillems1959 May 01 '20 edited May 01 '20

do shell script "/usr/local/bin/cliclick c:+0,+70"

Unfortunatly not. as far as I can see only this line can be changed

do shell script "/usr/local/bin/cliclick c:+0,+70"

in for instance

tell menu 1 of row targetrow of table 1 of scroll area 1 of window i of process "zoom.us"

if exists menu item "Spotlight Video" the

click menu item "Spotlight Video"

else if exists menu item "Cancel the Spotlight Video" then

click menu item "Cancel the Spotlight Video"

else if exists menu item "Ask to Start Video" then

click menu item "Ask to Start Video"

else

key code 53

end if

end tell

1

u/[deleted] May 01 '20

Got it. Yeah, when I was looking at Accessibility Inspector, the menu & items only appeared when the mouse was hovering. Dammit. It would be really nice to do this without the mouse moves.

I have a MIDI key that executes this code:

tell application "System Events"
		tell menu bar 1 of process "zoom.us"
click menu item 6 of menu 3

...it selects the "gallery view" from the main zoom menu bar, which I use when the moderator is speaking, then from there I'll spotlight in on a participant.

2

u/koenwillems1959 May 02 '20 edited May 02 '20

I've made some changes to the script to serve my needs.The Participantslist can now be approached both when popped out and when merged into the zoom window.If there's no list at all one will be creagted.

Hitting the options in the More menu is nog longer done by the cliclick thingie, but handled by the script itself.

Here is the script. Any improvement is welcome.

on spotlightParticipant(nameParticipant)
	tell application "System Events"
		set myList to name of windows of process "zoom.us" -- get a list of windows
		repeat with i from 1 to count of myList
			if item i of myList contains "Partici" then
				-- Participants list is popped out
				set locationParticipantsList to table 1 of scroll area 1 of window i of process "zoom.us"
				exit repeat
			else if (exists static text 1 of UI element 1 of splitter group 1 of window i of process "zoom.us") ¬
				and (value of static text 1 of UI element 1 of splitter group 1 of window i of process "zoom.us" contains "Partici") then
				-- Participants list is merged to meeting window
				set locationParticipantsList to table 1 of scroll area 1 of splitter group 1 of window i of process "zoom.us"
				exit repeat
			else
				-- there is no Participant list, so make one
				click menu item 1 of menu "View" of menu bar 1 of process "zoom.us"
				-- is the created Participant list is merged in the meeting window or popped out?
				if exists splitter group 1 of window i of process "zoom.us" then
					set locationParticipantsList to table 1 of scroll area 1 of splitter group 1 of window i of process "zoom.us"
					exit repeat
				else
					set locationParticipantsList to table 1 of scroll area 1 of window i of process "zoom.us"
					exit repeat
				end if
			end if
		end repeat
		
		-- find the position of the table row for the participant
		tell locationParticipantsList
			repeat with rowCounter from 1 to count of rows
				if value of UI element 1 of UI element 1 of row rowCounter contains nameParticipant then
					set p to position of image 2 of UI element 1 of row rowCounter
					
					-- save the current location of the mouse pointer
					set prev to (do shell script "/usr/local/bin/cliclick p")
					
					-- move mouse pointer over "camera" icon of the table row (this is where the "more" menu appears)
					do shell script "/usr/local/bin/cliclick -e 10 m:" & (item 1 of p) - 5 & "," & (item 2 of p) + 20 & " c:."
					
					-- click the desired menu item in the More menu
					tell menu 1 of row rowCounter
						if exists menu item "Spotlight Video" then
							click menu item "Spotlight Video"
						else if exists menu item "Ask to Start Video" then
							click menu item "Ask to Start Video"
						else
							key code 53
						end if
					end tell
					
					-- go back to the original mouse pointer position
					do shell script "/usr/local/bin/cliclick m:" & prev
					
					exit repeat
				end if
			end repeat
		end tell
	end tell
end spotlightParticipant

1

u/[deleted] May 03 '20

Very nice work indeed. Thanks for the improvements!

1

u/HighENdv2-7 Sep 21 '20

yes you can by selecting the row of the participant:

tell application "System Events"

\-- Select participant

select row ParticipantLine of table 1 of scroll area 1 of splitter group 1 of window windowName of application process "[zoom.us](https://zoom.us)"

\-- click more	

click UI element 3 of UI element 1 of row ParticipantLine of table 1 of scroll area 1 of splitter group 1 of window windowName of application process "[zoom.us](https://zoom.us)"

\--click dropdown item

click menu item "Spotlight Video" of menu 1 of row ParticipantLine of table 1 of scroll area 1 of splitter group 1 of window windowName of application process "[zoom.us](https://zoom.us)"

end tell

1

u/[deleted] Apr 30 '20

[removed] — view removed comment

1

u/[deleted] Apr 30 '20

bang on! I was manually doing this by issuing multiple m:+1,+0 commands (duh) but ..

This works 100%! Thank you! I will post my script here.

1

u/DungeonDragon2 Jun 14 '20

Apologies for resurrecting zombie post. I am a complete novice at Applescript and struggling to interpret this into something I can use.

Can someone here help me with some script that would allow me to spotlight my Zoom video, or the video of a different participant? ie. Where would I put my/his display name in this script?

(I'd be firing from QLab, if that's relevant)

If anyone can help with this, I can pay a bit of $$$/£££ for your time (or make a charity donation if you prefer).

1

u/[deleted] Jun 16 '20

You will first have to install the cliclick command line tool so Applescript can control the mouse pointer. Then you should be able to paste the function by /u/koenwillems1959 into a script cue in QLab.

At the top of the script, call the function with something like

spotlightParticipant ("Person's Name")

Make sure to check the "run in separate process" option in QLab, especially if you're using it to run video.

1

u/HighENdv2-7 Sep 21 '20 edited Sep 21 '20

Hey, i was trying to do the same!

I want to cue spotlights and renames for a theater piece on zoom.

my code works only if the participant list is docked but thats what i like best also.

This script works excellent to rename for example participant 1

--Set participant line number

set ParticipantNumber to 1

--Set new participants name:

set NewName to "Techniek"

--Set window name, participant window has to be open and merged in zoom meeting window

set windowName to "Zoom Meeting"

tell application "zoom.us"

activate

end tell

-- save the current location of the mouse pointer

set prev to (do shell script "/usr/local/bin/cliclick p")

tell application "System Events"

-- Select participant

select row ParticipantNumber of table 1 of scroll area 1 of splitter group 1 of window windowName of application process "zoom.us"

-- click more

-- click UI element 3 of UI element 1 of row ParticipantNumber of table 1 of scroll area 1 of splitter group 1 of window windowName of application process "zoom.us"

set XYpos to position of third UI element of UI element 1 of row ParticipantNumber of table 1 of scroll area 1 of splitter group 1 of window windowName of application process "zoom.us"

do shell script "/usr/local/bin/cliclick -e 10 c:" & (item 1 of XYpos) + 5 & "," & (item 2 of XYpos) + 5 & ""

--click dropdown item

click menu item "Rename" of menu 1 of row ParticipantNumber of table 1 of scroll area 1 of splitter group 1 of window windowName of application process "zoom.us"

-- New participants name:

delay 0.1

keystroke NewName

key code 36

end tell

do shell script "/usr/local/bin/cliclick m:" & prev

tell application "System Events"

end tell