Desktop Automation — Controlando a Tela com seu Agente | Controlling the Screen with Your Agent
-
Desktop Automation
PT-BR | EN
O que é
Desktop Automation permite que seu agente controle o mouse e teclado do seu computador como se fosse uma pessoa — capturando screenshots, movendo o cursor, clicando e digitando.
Ideal para automacao de tarefas repetitivas (RPA), testes de interface e tutoriais guiados.
Comandos Disponiveis
Comando Descricao Exemplo de Uso screenshot Captura a tela atual Tire um print da tela mouse_move Move o cursor para coordenadas (x,y) Mova o mouse para o canto superior esquerdo mouse_click Clica com botao esquerdo, direito ou duplo Clique no botao de enviar key_press Pressiona teclas ou combinacoes Pressione Ctrl+C ou Pressione Enter type_text Digita um texto automaticamente Digite ola mundo no campo de busca Exemplos Praticos
Capturar Tela
- Tire um print e me mostre
- Capture a tela para eu ver o que esta acontecendo
Navegar e Clicar
- Mova o mouse para a posicao 500, 300
- Clique com o botao direito
- De um duplo clique no icone
Digitar e Atalhos
- Digite 'meu texto aqui'
- Pressione Ctrl+Shift+Esc para abrir o Gerenciador de Tarefas
- Pressione Enter para confirmar
Casos de Uso
- RPA (Robotic Process Automation): Automatize tarefas repetitivas em sistemas legados
- Testes Automatizados: Teste interfaces graficas sem frameworks complexos
- Tutoriais Guiados: Mostre passo-a-passo ao usuario
- Suporte Remoto: Diagnostique problemas visualmente
Requisito: E necessario um display server ativo (funciona em Linux/macOS com ambiente grafico).
What It Is
Desktop Automation lets your agent control your computers mouse and keyboard just like a human — taking screenshots, moving the cursor, clicking, and typing.
Perfect for RPA (Robotic Process Automation), UI testing, and guided tutorials.
Available Commands
Command Description Example Usage screenshot Captures the current screen Take a screenshot and show me mouse_move Moves cursor to (x,y) coordinates Move the mouse to the top-left corner mouse_click Clicks left, right, or double Click the send button key_press Presses keys or combinations Press Ctrl+C or Press Enter type_text Types text automatically Type hello world in the search field Use Cases
- RPA: Automate repetitive tasks in legacy systems
- Automated Testing: Test GUIs without complex frameworks
- Guided Tutorials: Walk users through step-by-step processes
- Remote Support: Visually diagnose issues
Requirement: An active display server is needed (works on Linux/macOS with a graphical environment).
Dica: Combine screenshot + mouse_move + mouse_click para criar fluxos completos de automacao!
Tip: Combine screenshot + mouse_move + mouse_click to create complete automation flows! -
R Rodrigo Serpa moved this topic from Getting Started on
-
R Rodrigo Serpa moved this topic from Copy & Paste on
Hello! It looks like you're interested in this conversation, but you don't have an account yet.
Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.
With your input, this post could be even better 💗
Register Login