fix html template embed

Fastgif

README.md

@@ -6,22 +6,23 @@ A browser-in-browser "proxy" server that allows to use historical / vintage web
 
 ## Usage
 
-* [Download a WRP binary](https://github.com/tenox7/wrp/releases/) and run it on a machine that will become your WRP gateway/server. 
-This machine should be pretty modern, high spec and Google Chrome / Chromium Browser is required to be preinstalled.
-* Point your legacy browser to `http://address:port` of WRP server. Do not set or use it as a "proxy server".
-* Type a search string or a http/https URL and click **Go**.
+* [Download a WRP binary](https://github.com/tenox7/wrp/releases/) and run it on a machine that will become your WRP gateway/server. This machine should be pretty modern, high spec and Google Chrome / Chromium Browser is required to be preinstalled.
+* Make sure you don't have a firewall enabled or open up the port WRP is listening on (by default 8080).
+* Point your legacy browser to `http://address:port` of the WRP server. Do not set or use it as a "proxy server".
+* Type a search string or a full http/https URL and click **Go**.
 * Adjust your screen **W**idth/**H**eight/**S**cale/**C**olors to fit in your old browser.
 * Scroll web page by clicking on the in-image scroll bar.
+* WRP also allows **a single tall image without the vertical scrollbar** and use client scrolling. To enable this, simply height **H** to `0` . However this should not be used with old and low spec clients. Such tall images will be very large, take a lot of memory and long time to process, especially for GIFs.
 * Do not use client browser history-back, instead use **Bk** button in the app.
-* To send keystrokes, fill **K** input box and press **Go**. There also are buttons for backspace, enter and arrow keys.
-* You can set height **H** to `0` to render pages in to **a single tall image without the vertical scrollbar** and use client scrolling. However this should not be used with old and low spec clients. Such tall images will be very large and take long time to process, especially for GIFs.
-* Prefer PNG over GIF if your browser supports it. PNG is much faster, whereas GIF requires a lot of additional processing on both client and server.
 * You can re-capture page screenshot without reloading by using **St** (Stop).
 * You can also reload and re-capture current page with **Re** (Reload).
+* To send keystrokes, fill **K** input box and press **Go**. There also are buttons for backspace, enter and arrow keys.
+* Prefer PNG over GIF if your browser supports it. PNG is much faster, whereas GIF requires a lot of additional processing on both client and server to encode/decode.
+* GIF images are by default encoded with 216 colors, "web safe" palette. This uses an ultra fast but not very accurate color mapping algorithm. If you want better color representation switch to 256 color mode.
 
 ## Customization
 
-Since 4.5.1 WRP supports customizing it's own UI using HTML Template file. Download [wrp.html](wrp.html) place in the same directory with wrp binary customize it to your liking.
+WRP supports customizing it's own UI using HTML Template file. Download [wrp.html](wrp.html) place in the same directory with wrp binary customize it to your liking.
 
 ## Docker
 
@@ -49,16 +50,18 @@ Or from the [Azure Console](https://portal.azure.com/#create/Microsoft.Container
 
 Fortunately ACI allows port 80 without encryption.
 
-
 ## Flags
 
-```
--l  listen address:port (default :8080)
--t  image type gif or png (default gif) 
--g  image geometry, WxHxC, height can be 0 for unlimited (default 1152x600x256)
--h  headless mode, hide browser window on the server (default true)
--d  chromedp debug logging (default false)
--n  do not free maps and gif images after use (default false)
+```text
+-l   listen address:port (default :8080)
+-t   image type gif or png (default gif)
+-g   image geometry, WxHxC, height can be 0 for unlimited (default 1152x600x216)
+     C (number of colors) is only used for GIF
+-h   headless mode, hide browser window on the server (default true)
+-d   chromedp debug logging (default false)
+-n   do not free maps and gif images after use (default false)
+-ui  html template file (default "wrp.html")
+-s   delay/sleep after page is rendered before screenshot is taken (default 2s)
 ```
 
 ## UI explanation
@@ -117,8 +120,9 @@ $ ./wrp-amd64-macos -t png
 * In 2016 thanks to EFF/Certbot the whole internet migrated to HTTPS/SSL/TLS and WRP largely stopped working. Python code became unmaintainable and there was no easy way to make it work on Windows, even under WSL.
 * Version 3.0 (2019) has been rewritten in [Go](https://golang.org/) using [Chromedp](https://github.com/chromedp) as browser-in-browser instead of http-proxy. The initial version was [less than 100 lines of code](https://gist.github.com/tenox7/b0f03c039b0a8b67f6c1bf47e2dd0df0).
 * Version 4.0 has been completely refactored to use mouse clicks via imagemap instead parsing a href nodes. 
-* Version 4.1 added sending keystrokes in to input boxes. You can now login to Gmail. Also now runs as a Docker container and on Cloud Run/Azure Containers. 
+* Version 4.1 added sending keystrokes in to input boxes. You can now login to Gmail. Also now runs as a Docker container and on Cloud Run/Azure Containers.
 * Version 4.5 introduces rendering whole pages in to a single tall image with client scrolling.
+* Version 4.6 adds blazing fast gif encoding by [Hill Ma](https://github.com/mahiuchun).
 
 ## Credits
 
@@ -126,6 +130,7 @@ $ ./wrp-amd64-macos -t png
 * Uses [go-quantize](https://github.com/ericpauley/go-quantize), thanks to [ericpauley](https://github.com/ericpauley) for developing the missing go quantizer
 * Thanks to Jason Stevens of [Fun With Virtualization](https://virtuallyfun.com/) for graciously hosting my rumblings
 * Thanks to [claunia](https://github.com/claunia/) for help with the Python/Webkit version in the past
+* Thanks to [Hill Ma](https://github.com/mahiuchun) for ultra fast gif encoding algorithm
 * Historical Python/Webkit versions and prior art can be seen in [wrp-old](https://github.com/tenox7/wrp-old) repo
 
 ## Legal Stuff

go.mod

@@ -4,8 +4,14 @@ go 1.16
 
 require (
 	github.com/MaxHalford/halfgone v0.0.0-20171017091812-482157b86ccb
-	github.com/chromedp/cdproto v0.0.0-20221029224954-108014bf7279
+	github.com/chromedp/cdproto v0.0.0-20221126224343-3a0787b8dd28
 	github.com/chromedp/chromedp v0.8.6
-	github.com/ericpauley/go-quantize v0.0.0-20200331213906-ae555eb2afa4
-	golang.org/x/sys v0.1.0 // indirect
+	github.com/chromedp/sysutil v1.0.0 // indirect
+	github.com/gobwas/httphead v0.1.0 // indirect
+	github.com/gobwas/pool v0.2.1 // indirect
+	github.com/gobwas/ws v1.1.0 // indirect
+	github.com/josharian/intern v1.0.0 // indirect
+	github.com/mailru/easyjson v0.7.7 // indirect
+	github.com/soniakeys/quant v1.0.0
+	golang.org/x/sys v0.2.0 // indirect
 )

go.sum

@@ -3,12 +3,12 @@ github.com/MaxHalford/halfgone v0.0.0-20171017091812-482157b86ccb/go.mod h1:J86X
 github.com/chromedp/cdproto v0.0.0-20220924210414-0e3390be1777/go.mod h1:5Y4sD/eXpwrChIuxhSr/G20n9CdbCmoerOHnuAf0Zr0=
 github.com/chromedp/cdproto v0.0.0-20221029224954-108014bf7279 h1:7+D/pA8BoNzTpcM0Yw8issS95U/ipn0im5vzhfPzDZc=
 github.com/chromedp/cdproto v0.0.0-20221029224954-108014bf7279/go.mod h1:GKljq0VrfU4D5yc+2qA6OVr8pmO/MBbPEWqWQ/oqGEs=
+github.com/chromedp/cdproto v0.0.0-20221126224343-3a0787b8dd28 h1:i4vpMoaMguVwvDc0qSNbCHCRue6d0kbXjj5bDF4fHBA=
+github.com/chromedp/cdproto v0.0.0-20221126224343-3a0787b8dd28/go.mod h1:GKljq0VrfU4D5yc+2qA6OVr8pmO/MBbPEWqWQ/oqGEs=
 github.com/chromedp/chromedp v0.8.6 h1:KobeeqR2dpfKSG1prS3Y6+FbffMmGC6xmAobRXA9QEQ=
 github.com/chromedp/chromedp v0.8.6/go.mod h1:nBYHoD6YSNzrr82cIeuOzhw1Jo/s2o0QQ+ifTeoCZ+c=
 github.com/chromedp/sysutil v1.0.0 h1:+ZxhTpfpZlmchB58ih/LBHX52ky7w2VhQVKQMucy3Ic=
 github.com/chromedp/sysutil v1.0.0/go.mod h1:kgWmDdq8fTzXYcKIBqIYvRRTnYb9aNS9moAV0xufSww=
-github.com/ericpauley/go-quantize v0.0.0-20200331213906-ae555eb2afa4 h1:BBade+JlV/f7JstZ4pitd4tHhpN+w+6I+LyOS7B4fyU=
-github.com/ericpauley/go-quantize v0.0.0-20200331213906-ae555eb2afa4/go.mod h1:H7chHJglrhPPzetLdzBleF8d22WYOv7UM/lEKYiwlKM=
 github.com/gobwas/httphead v0.1.0 h1:exrUm0f4YX0L7EBwZHuCF4GDp8aJfVeBrlLQrs6NqWU=
 github.com/gobwas/httphead v0.1.0/go.mod h1:O/RXo79gxV8G+RqlR/otEwx4Q36zl9rqC5u12GKvMCM=
 github.com/gobwas/pool v0.2.1 h1:xfeeEhW7pwmX8nuLVlqbzVc7udMDrwetjEv+TZIz1og=
@@ -23,7 +23,11 @@ github.com/mailru/easyjson v0.7.7 h1:UGYAvKxe3sBsEDzO8ZeWOSlIQfWFlxbzLZe7hwFURr0
 github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc=
 github.com/orisano/pixelmatch v0.0.0-20220722002657-fb0b55479cde h1:x0TT0RDC7UhAVbbWWBzr41ElhJx5tXPWkIHA2HWPRuw=
 github.com/orisano/pixelmatch v0.0.0-20220722002657-fb0b55479cde/go.mod h1:nZgzbfBr3hhjoZnS66nKrHmduYNpc34ny7RK4z5/HM0=
+github.com/soniakeys/quant v1.0.0 h1:N1um9ktjbkZVcywBVAAYpZYSHxEfJGzshHCxx/DaI0Y=
+github.com/soniakeys/quant v1.0.0/go.mod h1:HI1k023QuVbD4H8i9YdfZP2munIHU4QpjsImz6Y6zds=
 golang.org/x/sys v0.0.0-20201207223542-d4d67f95c62d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20220928140112-f11e5e49a4ec/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.1.0 h1:kunALQeHf1/185U1i0GOB/fy1IPRDDpuoOOqRReG57U=
 golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.2.0 h1:ljd4t30dBnAvMZaQCevtY0xLLD0A+bRZXbgLMLU1F/A=
+golang.org/x/sys v0.2.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=

wrp.go

@@ -15,6 +15,7 @@ import (
 	"fmt"
 	"html/template"
 	"image"
+	"image/color/palette"
 	"image/gif"
 	"image/png"
 	"io/ioutil"
@@ -35,11 +36,11 @@ import (
 	"github.com/chromedp/cdproto/emulation"
 	"github.com/chromedp/cdproto/page"
 	"github.com/chromedp/chromedp"
-	"github.com/ericpauley/go-quantize/quantize"
+	"github.com/soniakeys/quant/median"
 )
 
 var (
-	version  = "4.5.3"
+	version  = "4.6.0"
 	srv      http.Server
 	ctx      context.Context
 	cancel   context.CancelFunc
@@ -49,9 +50,10 @@ var (
 	defType  string
 	defGeom  geom
 	htmlTmpl *template.Template
+	delay    time.Duration
 )
 
-// go:embed *.html
+//go:embed *.html
 var fs embed.FS
 
 type geom struct {
@@ -235,6 +237,45 @@ func chromedpCaptureScreenshot(res *[]byte, h int64) chromedp.Action {
 	})
 }
 
+func gifPalette(i image.Image, n int64) image.Image {
+	switch n {
+	case 2:
+		i = halfgone.FloydSteinbergDitherer{}.Apply(halfgone.ImageToGray(i))
+	case 216:
+		var FastGifLut = [256]int{0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5}
+		r := i.Bounds()
+		// NOTE: the color index computation below works only for palette.WebSafe!
+		p := image.NewPaletted(r, palette.WebSafe)
+		if i64, ok := i.(image.RGBA64Image); ok {
+			for y := r.Min.Y; y < r.Max.Y; y++ {
+				for x := r.Min.X; x < r.Max.X; x++ {
+					c := i64.RGBA64At(x, y)
+					r6 := FastGifLut[c.R>>8]
+					g6 := FastGifLut[c.G>>8]
+					b6 := FastGifLut[c.B>>8]
+					p.SetColorIndex(x, y, uint8(36*r6+6*g6+b6))
+				}
+			}
+		} else {
+			for y := r.Min.Y; y < r.Max.Y; y++ {
+				for x := r.Min.X; x < r.Max.X; x++ {
+					c := i.At(x, y)
+					r, g, b, _ := c.RGBA()
+					r6 := FastGifLut[r&0xff]
+					g6 := FastGifLut[g&0xff]
+					b6 := FastGifLut[b&0xff]
+					p.SetColorIndex(x, y, uint8(36*r6+6*g6+b6))
+				}
+			}
+		}
+		i = p
+	default:
+		q := median.Quantizer(n)
+		i = q.Paletted(i)
+	}
+	return i
+}
+
 // Capture currently rendered web page to an image and fake ISMAP
 func (rq *wrpReq) capture() {
 	var err error
@@ -266,7 +307,7 @@ func (rq *wrpReq) capture() {
 	}
 	chromedp.Run(
 		ctx, emulation.SetDeviceMetricsOverride(int64(float64(rq.width)/rq.zoom), height, rq.zoom, false),
-		chromedp.Sleep(time.Second*2), // TODO(tenox): totally lame, find a better way to determine if page is rendered
+		chromedp.Sleep(delay), // TODO(tenox): find a better way to determine if page is rendered
 	)
 	// Capture screenshot...
 	err = chromedp.Run(ctx, chromedpCaptureScreenshot(&pngcap, rq.height))
@@ -291,17 +332,13 @@ func (rq *wrpReq) capture() {
 	case "gif":
 		i, err := png.Decode(bytes.NewReader(pngcap))
 		if err != nil {
-			log.Printf("%s Failed to decode screenshot: %s\n", rq.r.RemoteAddr, err)
-			fmt.Fprintf(rq.w, "<BR>Unable to decode page screenshot:<BR>%s<BR>\n", err)
+			log.Printf("%s Failed to decode PNG screenshot: %s\n", rq.r.RemoteAddr, err)
+			fmt.Fprintf(rq.w, "<BR>Unable to decode page PNG screenshot:<BR>%s<BR>\n", err)
 			return
 		}
-		if rq.colors == 2 {
-			gray := halfgone.ImageToGray(i)
-			i = halfgone.FloydSteinbergDitherer{}.Apply(gray)
-		}
-		var gifbuf bytes.Buffer
 		st := time.Now()
-		err = gif.Encode(&gifbuf, i, &gif.Options{NumColors: int(rq.colors), Quantizer: quantize.MedianCutQuantizer{}})
+		var gifbuf bytes.Buffer
+		err = gif.Encode(&gifbuf, gifPalette(i, rq.colors), &gif.Options{})
 		if err != nil {
 			log.Printf("%s Failed to encode GIF: %s\n", rq.r.RemoteAddr, err)
 			fmt.Fprintf(rq.w, "<BR>Unable to encode GIF:<BR>%s<BR>\n", err)
@@ -311,7 +348,7 @@ func (rq *wrpReq) capture() {
 		ssize = fmt.Sprintf("%.0f KB", float32(len(gifbuf.Bytes()))/1024.0)
 		iw = i.Bounds().Max.X
 		ih = i.Bounds().Max.Y
-		log.Printf("%s Encoded GIF image: %s, Size: %s, Colors: %d, %dx%d, Time: %vms\n", rq.r.RemoteAddr, imgpath, ssize, rq.colors, iw, ih, time.Since(st).Milliseconds())
+		log.Printf("%s Encoded GIF image: %s, Size: %s, Colors: %d, Res: %dx%d, Time: %vms\n", rq.r.RemoteAddr, imgpath, ssize, rq.colors, iw, ih, time.Since(st).Milliseconds())
 	case "png":
 		pngbuf := bytes.NewBuffer(pngcap)
 		img[imgpath] = *pngbuf
@@ -319,7 +356,7 @@ func (rq *wrpReq) capture() {
 		ssize = fmt.Sprintf("%.0f KB", float32(len(pngbuf.Bytes()))/1024.0)
 		iw = cfg.Width
 		ih = cfg.Height
-		log.Printf("%s Got PNG image: %s, Size: %s, %dx%d\n", rq.r.RemoteAddr, imgpath, ssize, iw, ih)
+		log.Printf("%s Got PNG image: %s, Size: %s, Res: %dx%d\n", rq.r.RemoteAddr, imgpath, ssize, iw, ih)
 	}
 	rq.printHTML(printParams{
 		bgColor:    fmt.Sprintf("#%02X%02X%02X", r, g, b),
@@ -434,7 +471,7 @@ func tmpl(t string) string {
 	return string(tmpl)
 
 builtin:
-	fhs, err := fs.Open("/wrp.html")
+	fhs, err := fs.Open("wrp.html")
 	if err != nil {
 		log.Fatal(err)
 	}
@@ -458,8 +495,9 @@ func main() {
 	flag.BoolVar(&debug, "d", false, "Debug ChromeDP")
 	flag.BoolVar(&noDel, "n", false, "Do not free maps and images after use")
 	flag.StringVar(&defType, "t", "gif", "Image type: gif|png")
-	flag.StringVar(&fgeom, "g", "1152x600x256", "Geometry: width x height x colors, height can be 0 for unlimited")
+	flag.StringVar(&fgeom, "g", "1152x600x216", "Geometry: width x height x colors, height can be 0 for unlimited")
 	flag.StringVar(&tHTML, "ui", "wrp.html", "HTML template file for the UI")
+	flag.DurationVar(&delay, "s", 2*time.Second, "Delay/sleep after page is rendered and before screenshot is taken")
 	flag.Parse()
 	if len(os.Getenv("PORT")) > 0 {
 		addr = ":" + os.Getenv(("PORT"))

wrp.html

@@ -26,6 +26,7 @@
             </SELECT>
             C <SELECT NAME="c">
                 <OPTION VALUE="256" {{ if eq .NColors 256}}SELECTED{{end}}>256</OPTION>
+                <OPTION VALUE="216" {{ if eq .NColors 216}}SELECTED{{end}}>216</OPTION>
                 <OPTION VALUE="128" {{ if eq .NColors 128}}SELECTED{{end}}>128</OPTION>
                 <OPTION VALUE="64" {{ if eq .NColors 64}}SELECTED{{end}}>64</OPTION>
                 <OPTION VALUE="16" {{ if eq .NColors 16}}SELECTED{{end}}>16</OPTION>