Core Concepts

Mouse Control

Programmatically control the mouse cursor with precise movements, clicks, drags, and scrolling operations.

Try it out

Click anywhere to set a target, then run to move the cursor

cursor: (50, 50)

Code

typescript

import { mouse, straightTo, Point } from "@nut-tree/nut-js";

await mouse.move(straightTo(new Point(200, 150)));

What you can do

Precise Movement

Move the cursor to exact screen coordinates or relative positions

mouse.move(straightTo(new Point(100, 200)))

Click Actions

Left, right, middle clicks and double-clicks on any button

mouse.click(Button.LEFT)

Drag & Drop

Simulate drag operations from one point to another

mouse.drag(straightTo(new Point(400, 300)))

Scroll Control

Scroll up, down, left, or right by any amount

mouse.scrollDown(5)

Speed Control

Configure movement speed for smooth or instant actions

mouse.config.mouseSpeed = 1000

Position Reading

Get the current cursor position at any time

mouse.getPosition()

Quick Reference

move

mouse.move(path: Point[] | Promise<Point[]>)

Promise<void>

Move cursor along a path of points (use straightTo, centerOf, etc.)

click

mouse.click(button: Button)

Promise<void>

Perform a click with the specified button

doubleClick

mouse.doubleClick(button?: Button)

Promise<void>

Perform a double-click with the specified button

drag

mouse.drag(path: Point[] | Promise<Point[]>)

Promise<void>

Drag from current position along a path (use straightTo, etc.)

scrollDown

mouse.scrollDown(amount: number)

Promise<void>

Scroll down by the specified amount

getPosition

mouse.getPosition()

Promise<Point>

Get the current cursor position

Full API Reference

View complete mouse module documentation with all methods and options

Basic Movement

The mouse module provides methods for moving the cursor to absolute positions on screen. Use straightTo() to generate a straight-line path to a target point. All coordinates are in pixels, with (0, 0) at the top-left corner.

Move to Position

Move the cursor to an absolute screen position:

typescript

import { mouse, Point, straightTo } from "@nut-tree/nut-js";

// Move to coordinates using straightTo
await mouse.move(straightTo(new Point(100, 200)));

// Create a Point and move to it
const target = new Point(500, 300);
await mouse.move(straightTo(target));

Get Current Position

Read the current cursor position:

typescript

const position = await mouse.getPosition();
console.log(`Cursor at: ${position.x}, ${position.y}`);

Clicking

nut.js supports all standard mouse buttons. Use the Button enum for explicit button selection.

typescript

import { mouse, Button } from "@nut-tree/nut-js";

// Left click at current position
await mouse.click(Button.LEFT);

// Right click
await mouse.click(Button.RIGHT);

// Middle click
await mouse.click(Button.MIDDLE);

// Double click
await mouse.doubleClick(Button.LEFT);

Dragging

Perform drag operations by specifying a path to the target. The drag starts from the current cursor position.

typescript

import { mouse, Point, straightTo } from "@nut-tree/nut-js";

// Drag from current position to target
await mouse.drag(straightTo(new Point(400, 300)));

// Or move first, then drag
await mouse.move(straightTo(new Point(100, 100)));
await mouse.drag(straightTo(new Point(500, 500)));

Scrolling

Scroll the mouse wheel vertically or horizontally:

typescript

import { mouse } from "@nut-tree/nut-js";

// Scroll down
await mouse.scrollDown(5);

// Scroll up
await mouse.scrollUp(3);

// Scroll left/right (where supported)
await mouse.scrollLeft(2);
await mouse.scrollRight(2);

Configuration

Configure mouse movement speed and behavior:

typescript

import { mouse } from "@nut-tree/nut-js";

// Set movement speed (pixels per second)
mouse.config.mouseSpeed = 1000;

// Disable automatic delays between actions
mouse.config.autoDelayMs = 0;

Pro Tip

For more reliable automation, consider using image search to find UI elements rather than hardcoded coordinates. See the Screen & Image Search guide.

Custom Mouse Movement

While nut.js provides built-in movement functions like straight-line movement, you can create custom movement functions for more natural or creative cursor paths. The mouse.move() method accepts an array of points, allowing you to define any path you want.

Sine Wave Movement

This example creates a function that moves the cursor along a sine wave path instead of a straight line. This can make movements appear more natural or help avoid detection in certain automation scenarios.

typescript

import { mouse, Point } from "@nut-tree/nut-js";

async function alongSineWaveTo(
  endPoint: Point | Promise<Point>,
  amplitude = 50,
  frequency = 4,
  numPoints = 1000
): Promise<Point[]> {
  const source = await mouse.getPosition();
  const target = await endPoint;

  // Calculate slope and length of the line segment
  const dx = target.x - source.x;
  const dy = target.y - source.y;
  const length = Math.sqrt(dx * dx + dy * dy);

  // Normal vector (perpendicular to the line segment)
  const nx = -dy / length;
  const ny = dx / length;

  const sineWavePoints: Point[] = [];
  for (let idx = 0; idx <= numPoints; idx++) {
    const t = idx / numPoints;
    const waveAmplitude = amplitude * Math.sin(2 * Math.PI * frequency * t);

    // Project the sine wave perpendicular to the path
    const x = source.x + dx * t + nx * waveAmplitude;
    const y = source.y + dy * t + ny * waveAmplitude;

    sineWavePoints.push(new Point(x, y));
  }

  return sineWavePoints;
}

Using Custom Movement

Use your custom movement function by passing its result to mouse.move():

typescript

// Move to a specific point along a sine wave
await mouse.move(alongSineWaveTo(new Point(500, 300)));

// Customize the wave parameters
await mouse.move(alongSineWaveTo(
  new Point(800, 400),
  30,   // amplitude - height of the wave
  6,    // frequency - number of wave cycles
  500   // numPoints - smoothness of the path
));

// Combine with image search for dynamic targets
await mouse.move(
  alongSineWaveTo(
    centerOf(screen.find(imageResource("button.png")))
  )
);

Movement Function Parameters

amplitude - The height of the sine wave perpendicular to the path (default: 50px)
frequency - Number of complete wave cycles (default: 4)
numPoints - Number of points in the path, higher = smoother (default: 1000)

Other Movement Patterns

You can create any movement pattern by generating an array of points. Here are some ideas:

typescript

import { mouse, Point } from "@nut-tree/nut-js";

// Bezier curve movement
async function alongBezierTo(endPoint: Point, controlPoint: Point) {
  const start = await mouse.getPosition();
  const end = await endPoint;
  const points: Point[] = [];

  for (let t = 0; t <= 1; t += 0.01) {
    const x = (1 - t) ** 2 * start.x + 2 * (1 - t) * t * controlPoint.x + t ** 2 * end.x;
    const y = (1 - t) ** 2 * start.y + 2 * (1 - t) * t * controlPoint.y + t ** 2 * end.y;
    points.push(new Point(x, y));
  }

  return points;
}

// Spiral movement inward to target
async function spiralTo(endPoint: Point, turns = 3) {
  const start = await mouse.getPosition();
  const end = await endPoint;
  const points: Point[] = [];
  const numPoints = 200;

  for (let i = 0; i <= numPoints; i++) {
    const t = i / numPoints;
    const angle = turns * 2 * Math.PI * t;
    const radius = (1 - t) * 50; // Spiral inward

    const baseX = start.x + (end.x - start.x) * t;
    const baseY = start.y + (end.y - start.y) * t;

    points.push(new Point(
      baseX + Math.cos(angle) * radius,
      baseY + Math.sin(angle) * radius
    ));
  }

  return points;
}

Real-World Examples

Here are some practical examples showing how to use mouse control in real automation scenarios.

Automate Form Filling

Click through a multi-step form and fill in fields

Scenario: You need to automate a web form that requires clicking different input fields and navigating between pages.

typescript

import { mouse, keyboard, Button, Key, Point, straightTo } from "@nut-tree/nut-js";

// Click the username field
await mouse.move(straightTo(new Point(400, 200)));
await mouse.click(Button.LEFT);
await keyboard.type("john.doe@example.com");

// Tab to password field
await keyboard.pressKey(Key.Tab);
await keyboard.type("secretpassword");

// Click submit button
await mouse.move(straightTo(new Point(400, 350)));
await mouse.click(Button.LEFT);

Drag Files Between Folders

Select and drag files from one location to another

Scenario: You want to organize files by dragging them from a downloads folder to a project folder in a file manager.

typescript

import { mouse, keyboard, Button, Key, Point, straightTo } from "@nut-tree/nut-js";

// Select the file
await mouse.move(straightTo(new Point(200, 300)));
await mouse.click(Button.LEFT);

// Drag to destination folder
await mouse.drag(straightTo(new Point(600, 300)));

// For multiple files, hold Ctrl and click each
await mouse.move(straightTo(new Point(200, 300)));
await keyboard.pressKey(Key.LeftControl);
await mouse.click(Button.LEFT);
await mouse.move(straightTo(new Point(200, 340)));
await mouse.click(Button.LEFT);
await keyboard.releaseKey(Key.LeftControl);

// Then drag all selected
await mouse.drag(straightTo(new Point(600, 300)));

Scroll Through Content

Navigate long pages or lists by scrolling

Scenario: You need to scroll through a long webpage to find and click on a specific element that's below the fold.

typescript

import { mouse, Button, Point, straightTo } from "@nut-tree/nut-js";

// Move to the scrollable area first
await mouse.move(straightTo(new Point(500, 400)));

// Scroll down to reveal more content
for (let i = 0; i < 5; i++) {
  await mouse.scrollDown(1);
}

// Wait a moment for content to load
await new Promise(r => setTimeout(r, 500));

// Now click on the revealed element
await mouse.move(straightTo(new Point(500, 300)));
await mouse.click(Button.LEFT);