Skip to content

Commit 67710b7

Browse files
Add --headless flag to start HTTP server without TUI (#27)
* feat: implement headless server mode with direct download handling and configurable output paths. * feat: clarify headless mode usage, output directory configuration, and download path hierarchy in README, and update benchmark script. * updated readme * Enhance README with operational modes and optimizations Added details about operational modes and how Surge works.
1 parent b3753c9 commit 67710b7

5 files changed

Lines changed: 280 additions & 62 deletions

File tree

‎README.md‎

Lines changed: 107 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,9 @@
66
[![Stars](https://img.shields.io/github/stars/surge-downloader/surge?style=flat-square)](https://github.com/surge-downloader/surge/stargazers)
77
[![Last Commit](https://img.shields.io/github/last-commit/surge-downloader/surge?style=flat-square)](https://github.com/surge-downloader/surge/commits/main)
88

9-
Surge is a blazing fast, open-source terminal (TUI) download manager built in Go. Designed for power users who prefer a keyboard-driven workflow and want full control over their downloads.
9+
Surge is a blazing fast, open-source download manager built in Go. While it features a beautiful **Terminal User Interface (TUI)**, it is architected to run equally well as a background **Headless Server** or a **CLI tool** for automation.
10+
11+
Designed for power users who prefer a keyboard-driven workflow and want full control over their downloads.
1012

1113
![demo](assets/demo.gif)
1214

@@ -28,6 +30,7 @@ brew install surge-downloader/tap/surge
2830

2931
```bash
3032
go install github.com/surge-downloader/surge@latest
33+
3134
```
3235

3336
### Build from Source
@@ -36,42 +39,123 @@ go install github.com/surge-downloader/surge@latest
3639
git clone https://github.com/surge-downloader/surge.git
3740
cd surge
3841
go build -o surge .
42+
3943
```
4044

41-
## Features
45+
---
46+
<details><summary>
47+
<h2> Operational Modes </h2>
48+
</summary>
49+
Surge operates in three distinct modes depending on your workflow needs:
50+
51+
### 1. TUI Mode (Interactive)
52+
53+
The default mode. Launches a full-screen, interactive dashboard to manage downloads, view real-time graphs, and manage the queue.
54+
55+
* **Best for:** Daily usage, visual monitoring.
56+
57+
### 2. Headless Mode (Server/Daemon)
58+
59+
Runs Surge in the background without a user interface. It listens on a specific port for incoming download requests from the CLI or Browser Extension.
60+
61+
* **Best for:** Raspberry Pis, VPS, always-on servers, or background tasks.
62+
63+
### 3. CLI Mode (Client & Automation)
64+
65+
Allows you to send commands to a running instance (TUI or Headless) or perform quick, standalone downloads similar to `wget` or `curl`.
4266

43-
- **High-speed Downloads** with multi-connection support
44-
- **Beautiful TUI** built with Bubble Tea & Lipgloss
45-
- **Pause/Resume** downloads seamlessly
46-
- **Real-time Progress** with speed graphs and ETA
47-
- **Auto-retry** on connection failures
48-
- **Batch Downloads**
49-
- **Browser Extension** integration
50-
- **Clipboard Integration**
67+
* **Best for:** Scripts, batch processing, single-file downloads.
68+
</details>
69+
70+
---
5171

5272
## Usage
5373

74+
### Interactive TUI
75+
76+
Start the visual dashboard.
77+
5478
```bash
55-
# Start TUI mode
5679
surge
5780

58-
# Headless download (CLI only, no TUI)
59-
surge get <URL>
81+
```
6082

61-
# Headless download with custom output directory
62-
surge get <URL> -o ~/Downloads
83+
### Headless Server
84+
85+
Start the daemon in the background.
86+
87+
```bash
88+
# Start server (default port)
89+
surge --headless
90+
91+
# Start with specific settings
92+
surge --headless --port 8090 -o ~/Downloads/Surge
93+
94+
```
6395

64-
# Send download via CLI to already running TUI instance
65-
surge get <URL> --port <PORT>
96+
> **Note:** Set port to `0` to have Surge automatically assign a free port.
6697
67-
# Batch download from a file (one URL per line)
98+
### CLI & Remote Control
99+
100+
Send downloads to a running Surge instance (TUI or Headless) or download directly.
101+
102+
```bash
103+
# 1. Send to a running instance (Remote)
104+
# Does not block your terminal; hands off the task to the server.
105+
surge get <URL> --port 8090
106+
107+
# 2. Standalone Download (Direct)
108+
# Blocks the terminal until finished (like wget). No server required.
109+
surge get <URL>
110+
111+
# 3. Override Output Directory
112+
surge get <URL> -o /tmp/downloads
113+
114+
# 4. Batch Download
115+
# Reads a file with one URL per line
68116
surge get --batch urls.txt
117+
69118
```
70119

120+
---
121+
122+
## Features
123+
124+
* **High-speed Downloads** with multi-connection support
125+
* **Beautiful TUI** built with Bubble Tea & Lipgloss
126+
* **Pause/Resume** downloads seamlessly
127+
* **Real-time Progress** with speed graphs and ETA
128+
* **Auto-retry** on connection failures
129+
* **Batch Downloads**
130+
* **Browser Extension** integration
131+
* **Clipboard Integration**
132+
133+
---
134+
135+
<details>
136+
<summary><h2> How it Works? </h2></summary>
137+
138+
A standard browser usually opens a single HTTP connection to the server. However, servers typically limit the bandwidth allocated to a single connection to ensure fairness for all users.
139+
140+
Download managers (like Surge) open multiple requests simultaneously (e.g., 32 in Surge). They use this method to split the file into many small parts and download those parts individually.
141+
142+
Not all connections are created equal; there are fast connections and slow connections due to factors like load balancers and CDNs. Download managers employ various methods to optimize these connections.
143+
144+
The top 3 optimizations we did in Surge are:
145+
146+
1. **Split the Largest Chunk:** Split the largest chunk whenever possible so that workers do not remain idle.
147+
148+
2. **Work Stealing:** Near the end, when fast workers are finished and slow workers are still processing, make the fast, idle workers "steal work" from the slow workers.
149+
150+
3. **Restart Slow Workers:** Calculate the mean speed of all workers. If a worker is performing at less than **0.3x** of the mean, restart it in the hopes that it will secure a better pathway to the server, which will be faster.
151+
</details>
152+
153+
---
154+
71155
## Benchmarks
72156

73157
| Tool | Time | Speed | vs Surge |
74-
|------|------|-------|----------|
158+
| --- | --- | --- | --- |
75159
| **Surge** | 28.93s | **35.40 MB/s** | — |
76160
| aria2c | 40.04s | 25.57 MB/s | 1.38× slower |
77161
| curl | 57.57s | 17.79 MB/s | 1.99× slower |
@@ -82,15 +166,16 @@ surge get --batch urls.txt
82166

83167
*Results averaged over 5 runs*
84168

85-
| | |
86-
|---|---|
169+
| | |
170+
| --- | --- |
87171
| **File** | 1GB.bin ([link](https://sin-speed.hetzner.com/1GB.bin)) |
88172
| **OS** | Windows 11 Pro |
89173
| **CPU** | AMD Ryzen 5 5600X |
90174
| **RAM** | 16 GB DDR4 |
91175
| **Network** | 360 Mbps / 45 MB/s |
92176

93177
Run your own: `python benchmark.py -n 5`
178+
94179
</details>
95180

96181
## Browser Extension
@@ -123,4 +208,4 @@ Contributions are welcome! Feel free to fork, make changes, and submit a pull re
123208

124209
If you find Surge useful, please consider giving it a ⭐ it helps others discover the project!
125210

126-
This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.
211+
This project is licensed under the MIT License. See the [LICENSE](https://www.google.com/search?q=LICENSE) file for details.

‎benchmark.py‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -159,8 +159,6 @@ def benchmark_surge(project_dir: Path, url: str, output_dir: Path) -> BenchmarkR
159159
success, output = run_command([
160160
str(surge_bin), "get", url,
161161
"--output", str(output_dir), # Download directory
162-
# "--headless", # Removed as it's default/unrecognized
163-
# "-c", "16", # Removed as concurrent defaults are dynamic or strictly internal
164162
], timeout=600)
165163
elapsed = time.perf_counter() - start
166164

‎cmd/cmd_test.go‎

Lines changed: 16 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -181,7 +181,7 @@ func TestHandleDownload_MethodNotAllowed(t *testing.T) {
181181
req := httptest.NewRequest(http.MethodGet, "/download", nil)
182182
rec := httptest.NewRecorder()
183183

184-
handleDownload(rec, req)
184+
handleDownload(rec, req, "")
185185

186186
if rec.Code != http.StatusMethodNotAllowed {
187187
t.Errorf("Expected 405, got %d", rec.Code)
@@ -192,7 +192,7 @@ func TestHandleDownload_InvalidJSON(t *testing.T) {
192192
req := httptest.NewRequest(http.MethodPost, "/download", bytes.NewBufferString("not json"))
193193
rec := httptest.NewRecorder()
194194

195-
handleDownload(rec, req)
195+
handleDownload(rec, req, "")
196196

197197
if rec.Code != http.StatusBadRequest {
198198
t.Errorf("Expected 400, got %d", rec.Code)
@@ -207,7 +207,7 @@ func TestHandleDownload_MissingURL(t *testing.T) {
207207
req := httptest.NewRequest(http.MethodPost, "/download", bytes.NewBufferString(body))
208208
rec := httptest.NewRecorder()
209209

210-
handleDownload(rec, req)
210+
handleDownload(rec, req, "")
211211

212212
if rec.Code != http.StatusBadRequest {
213213
t.Errorf("Expected 400, got %d", rec.Code)
@@ -222,7 +222,7 @@ func TestHandleDownload_EmptyURL(t *testing.T) {
222222
req := httptest.NewRequest(http.MethodPost, "/download", bytes.NewBufferString(body))
223223
rec := httptest.NewRecorder()
224224

225-
handleDownload(rec, req)
225+
handleDownload(rec, req, "")
226226

227227
if rec.Code != http.StatusBadRequest {
228228
t.Errorf("Expected 400, got %d", rec.Code)
@@ -245,7 +245,7 @@ func TestHandleDownload_PathTraversal(t *testing.T) {
245245
t.Run(tt.name, func(t *testing.T) {
246246
req := httptest.NewRequest(http.MethodPost, "/download", bytes.NewBufferString(tt.body))
247247
rec := httptest.NewRecorder()
248-
handleDownload(rec, req)
248+
handleDownload(rec, req, "")
249249

250250
if rec.Code != http.StatusBadRequest {
251251
t.Errorf("Expected 400, got %d", rec.Code)
@@ -511,7 +511,7 @@ func TestStartHTTPServer_HealthEndpoint(t *testing.T) {
511511
port := ln.Addr().(*net.TCPAddr).Port
512512

513513
// Start server in background
514-
go startHTTPServer(ln, port)
514+
go startHTTPServer(ln, port, "")
515515

516516
// Give server time to start
517517
time.Sleep(50 * time.Millisecond)
@@ -547,7 +547,7 @@ func TestStartHTTPServer_NoCORSHeaders(t *testing.T) {
547547
}
548548
port := ln.Addr().(*net.TCPAddr).Port
549549

550-
go startHTTPServer(ln, port)
550+
go startHTTPServer(ln, port, "")
551551
time.Sleep(50 * time.Millisecond)
552552

553553
resp, err := http.Get(fmt.Sprintf("http://127.0.0.1:%d/health", port))
@@ -568,7 +568,7 @@ func TestStartHTTPServer_OptionsRequest(t *testing.T) {
568568
}
569569
port := ln.Addr().(*net.TCPAddr).Port
570570

571-
go startHTTPServer(ln, port)
571+
go startHTTPServer(ln, port, "")
572572
time.Sleep(50 * time.Millisecond)
573573

574574
req, _ := http.NewRequest(http.MethodOptions, fmt.Sprintf("http://127.0.0.1:%d/download", port), nil)
@@ -590,7 +590,7 @@ func TestStartHTTPServer_DownloadEndpoint_MethodNotAllowed(t *testing.T) {
590590
}
591591
port := ln.Addr().(*net.TCPAddr).Port
592592

593-
go startHTTPServer(ln, port)
593+
go startHTTPServer(ln, port, "")
594594
time.Sleep(50 * time.Millisecond)
595595

596596
// GET should not be allowed
@@ -612,7 +612,7 @@ func TestStartHTTPServer_DownloadEndpoint_BadRequest(t *testing.T) {
612612
}
613613
port := ln.Addr().(*net.TCPAddr).Port
614614

615-
go startHTTPServer(ln, port)
615+
go startHTTPServer(ln, port, "")
616616
time.Sleep(50 * time.Millisecond)
617617

618618
// POST with invalid JSON
@@ -638,7 +638,7 @@ func TestStartHTTPServer_DownloadEndpoint_MissingURL(t *testing.T) {
638638
}
639639
port := ln.Addr().(*net.TCPAddr).Port
640640

641-
go startHTTPServer(ln, port)
641+
go startHTTPServer(ln, port, "")
642642
time.Sleep(50 * time.Millisecond)
643643

644644
// POST with missing URL
@@ -664,7 +664,7 @@ func TestStartHTTPServer_NotFoundEndpoint(t *testing.T) {
664664
}
665665
port := ln.Addr().(*net.TCPAddr).Port
666666

667-
go startHTTPServer(ln, port)
667+
go startHTTPServer(ln, port, "")
668668
time.Sleep(50 * time.Millisecond)
669669

670670
resp, err := http.Get(fmt.Sprintf("http://127.0.0.1:%d/nonexistent", port))
@@ -702,14 +702,14 @@ func TestHandleDownload_ValidRequest_NoServerProgram(t *testing.T) {
702702
}
703703
}()
704704

705-
handleDownload(rec, req)
705+
handleDownload(rec, req, "")
706706
}
707707

708708
func TestHandleDownload_EmptyBody(t *testing.T) {
709709
req := httptest.NewRequest(http.MethodPost, "/download", bytes.NewBufferString(""))
710710
rec := httptest.NewRecorder()
711711

712-
handleDownload(rec, req)
712+
handleDownload(rec, req, "")
713713

714714
// Empty body causes EOF error on decode
715715
if rec.Code != http.StatusBadRequest {
@@ -725,7 +725,7 @@ func TestHandleDownload_LargeURL(t *testing.T) {
725725
rec := httptest.NewRecorder()
726726

727727
// This should handle large URLs gracefully (validation issues)
728-
handleDownload(rec, req)
728+
handleDownload(rec, req, "")
729729

730730
// Should fail on URL validation or JSON parsing
731731
t.Logf("Response: %d", rec.Code)
@@ -742,7 +742,7 @@ func TestHandleDownload_SpecialCharactersInPath(t *testing.T) {
742742
}
743743
}()
744744

745-
handleDownload(rec, req)
745+
handleDownload(rec, req, "")
746746
}
747747

748748
// =============================================================================

‎cmd/get.go‎

Lines changed: 26 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ import (
1212
"strings"
1313
"time"
1414

15+
"github.com/surge-downloader/surge/internal/config"
1516
"github.com/surge-downloader/surge/internal/download"
1617
"github.com/surge-downloader/surge/internal/messages"
1718
"github.com/surge-downloader/surge/internal/utils"
@@ -98,7 +99,15 @@ func runHeadless(ctx context.Context, url, outPath string, verbose bool) error {
9899
}
99100
}
100101

101-
return <-errCh
102+
err := <-errCh
103+
if err == nil {
104+
elapsed := time.Since(startTime)
105+
speed := float64(totalSize) / elapsed.Seconds() / (1024 * 1024)
106+
fmt.Fprintf(os.Stderr, "Complete: %s in %s (%.2f MB/s)\n",
107+
utils.ConvertBytesToHumanReadable(totalSize),
108+
elapsed.Round(time.Millisecond), speed)
109+
}
110+
return err
102111
}
103112

104113
// sendToServer sends a download request to a running surge server
@@ -133,7 +142,6 @@ var getCmd = &cobra.Command{
133142
Short: "Download a file in headless mode or send to running server",
134143
Long: `Download a file from a URL without the TUI interface.
135144
136-
Use --headless for CLI-only downloads (useful for scripting).
137145
Use --port to send the download to a running Surge instance.
138146
Use --batch to download multiple URLs from a file (one URL per line).`,
139147
Args: cobra.MaximumNArgs(1),
@@ -181,9 +189,22 @@ Use --batch to download multiple URLs from a file (one URL per line).`,
181189
}
182190

183191
if outPath == "" && port == 0 {
184-
// Only default to "." for headless mode.
185-
// For server mode (port > 0), send empty path so TUI uses its default.
186-
outPath = "."
192+
// Try to load default download directory from settings
193+
settings, err := config.LoadSettings()
194+
if err == nil && settings.General.DefaultDownloadDir != "" {
195+
outPath = settings.General.DefaultDownloadDir
196+
// Create directory if it doesn't exist
197+
if err := os.MkdirAll(outPath, 0755); err != nil {
198+
// If creation fails, fallback to current directory
199+
outPath = "."
200+
}
201+
} else {
202+
// Fallback to current directory
203+
outPath = "."
204+
}
205+
} else if outPath == "" && port > 0 {
206+
// For server mode, send empty path so server/TUI uses its default
207+
outPath = ""
187208
}
188209

189210
// Process each URL

0 commit comments

Comments
 (0)