Server-side syntax highlighting for all code (#12047)

* Server-side syntax hilighting for all code

This PR does a few things:

* Remove all traces of highlight.js
* Use chroma library to provide fast syntax hilighting directly on the server
* Provide syntax hilighting for diffs
* Re-style both unified and split diffs views
* Add custom syntax hilighting styling for both regular and arc-green

Fixes #7729
Fixes #10157
Fixes #11825
Fixes #7728
Fixes #3872
Fixes #3682

And perhaps gets closer to #9553

* fix line marker

* fix repo search

* Fix single line select

* properly load settings

* npm uninstall highlight.js

* review suggestion

* code review

* forgot to call function

* fix test

* Apply suggestions from code review

suggestions from @silverwind thanks

Co-authored-by: silverwind <me@silverwind.io>

* code review

* copy/paste error

* Use const for highlight size limit

* Update web_src/less/_repository.less

Co-authored-by: Lauris BH <lauris@nix.lv>

* update size limit to 1MB and other styling tweaks

* fix highlighting for certain diff sections

* fix test

* add worker back as suggested

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Lauris BH <lauris@nix.lv>
mj-v1.14.3
mrsdizzie 3 years ago committed by GitHub
parent ce5f2b9845
commit af7ffaa279
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

@ -20,6 +20,7 @@ require (
github.com/BurntSushi/toml v0.3.1
github.com/PuerkitoBio/goquery v1.5.0
github.com/RoaringBitmap/roaring v0.4.23 // indirect
github.com/alecthomas/chroma v0.7.3
github.com/bgentry/speakeasy v0.1.0 // indirect
github.com/blevesearch/bleve v1.0.7
github.com/couchbase/gomemcached v0.0.0-20191004160342-7b5da2ec40b2 // indirect
@ -66,7 +67,7 @@ require (
github.com/lunny/dingtalk_webhook v0.0.0-20171025031554-e3534c89ef96
github.com/mailru/easyjson v0.7.0 // indirect
github.com/markbates/goth v1.61.2
github.com/mattn/go-isatty v0.0.11
github.com/mattn/go-isatty v0.0.12
github.com/mattn/go-sqlite3 v2.0.2+incompatible
github.com/mcuadros/go-version v0.0.0-20190308113854-92cdf37c5b75
github.com/mgechev/dots v0.0.0-20190921121421-c36f7dcfbb81
@ -99,6 +100,7 @@ require (
github.com/urfave/cli v1.20.0
github.com/xanzy/go-gitlab v0.31.0
github.com/yohcop/openid-go v1.0.0
github.com/yuin/goldmark-highlighting v0.0.0-20200307114337-60d527fdb691
github.com/yuin/goldmark v1.1.32
github.com/yuin/goldmark-meta v0.0.0-20191126180153-f0638e958b60
golang.org/x/crypto v0.0.0-20200604202706-70a84ac30bf9

@ -46,6 +46,8 @@ gitea.com/xorm/sqlfiddle v0.0.0-20180821085327-62ce714f951a/go.mod h1:EXuID2Zs0p
github.com/BurntSushi/toml v0.3.1 h1:WXkYYl6Yr3qBf1K79EBnL4mak0OimBfB0XUf9Vl28OQ=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo=
github.com/GeertJohan/go.incremental v1.0.0/go.mod h1:6fAjUhbVuX1KcMD3c8TEgVUqmo4seqhv0i0kdATSkM0=
github.com/GeertJohan/go.rice v1.0.0/go.mod h1:eH6gbSOAUv07dQuZVnBmoDP8mgsM1rtixis4Tib9if0=
github.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU=
github.com/PuerkitoBio/goquery v1.5.0 h1:uGvmFXOA73IKluu/F84Xd1tt/z07GYm8X49XKHP7EJk=
github.com/PuerkitoBio/goquery v1.5.0/go.mod h1:qD2PgZ9lccMbQlc7eEOjaeRlFQON7xY8kdmcsrnKqMg=
@ -61,8 +63,22 @@ github.com/RoaringBitmap/roaring v0.4.23/go.mod h1:D0gp8kJQgE1A4LQ5wFLggQEyvDi06
github.com/Shopify/sarama v1.19.0/go.mod h1:FVkBWblsNy7DGZRfXLU0O9RCGt5g3g3yEuWXgklEdEo=
github.com/Shopify/toxiproxy v2.1.4+incompatible/go.mod h1:OXgGpZ6Cli1/URJOF1DMxUHB2q5Ap20/P/eIdh4G0pI=
github.com/Unknwon/com v0.0.0-20190321035513-0fed4efef755/go.mod h1:voKvFVpXBJxdIPeqjoJuLK+UVcRlo/JLjeToGxPYu68=
github.com/akavel/rsrc v0.8.0/go.mod h1:uLoCtb9J+EyAqh+26kdrTgmzRBFPGOolLWKpdxkKq+c=
github.com/alcortesm/tgz v0.0.0-20161220082320-9c5fe88206d7 h1:uSoVVbwJiQipAclBbw+8quDsfcvFjOpI5iCf4p/cqCs=
github.com/alcortesm/tgz v0.0.0-20161220082320-9c5fe88206d7/go.mod h1:6zEj6s6u/ghQa61ZWa/C2Aw3RkjiTBOix7dkqa1VLIs=
github.com/alecthomas/assert v0.0.0-20170929043011-405dbfeb8e38 h1:smF2tmSOzy2Mm+0dGI2AIUHY+w0BUc+4tn40djz7+6U=
github.com/alecthomas/assert v0.0.0-20170929043011-405dbfeb8e38/go.mod h1:r7bzyVFMNntcxPZXK3/+KdruV1H5KSlyVY0gc+NgInI=
github.com/alecthomas/chroma v0.7.2-0.20200305040604-4f3623dce67a/go.mod h1:fv5SzZPFJbwp2NXJWpFIX7DZS4HgV1K4ew4Pc2OZD9s=
github.com/alecthomas/chroma v0.7.3 h1:NfdAERMy+esYQs8OXk0I868/qDxxCEo7FMz1WIqMAeI=
github.com/alecthomas/chroma v0.7.3/go.mod h1:sko8vR34/90zvl5QdcUdvzL3J8NKjAUx9va9jPuFNoM=
github.com/alecthomas/colour v0.0.0-20160524082231-60882d9e2721 h1:JHZL0hZKJ1VENNfmXvHbgYlbUOvpzYzvy2aZU5gXVeo=
github.com/alecthomas/colour v0.0.0-20160524082231-60882d9e2721/go.mod h1:QO9JBoKquHd+jz9nshCh40fOfO+JzsoXy8qTHF68zU0=
github.com/alecthomas/kong v0.1.17-0.20190424132513-439c674f7ae0/go.mod h1:+inYUSluD+p4L8KdviBSgzcqEjUQOfC5fQDRFuc36lI=
github.com/alecthomas/kong v0.2.1-0.20190708041108-0548c6b1afae/go.mod h1:+inYUSluD+p4L8KdviBSgzcqEjUQOfC5fQDRFuc36lI=
github.com/alecthomas/kong v0.2.4/go.mod h1:kQOmtJgV+Lb4aj+I2LEn40cbtawdWJ9Y8QLq+lElKxE=
github.com/alecthomas/kong-hcl v0.1.8-0.20190615233001-b21fea9723c8/go.mod h1:MRgZdU3vrFd05IQ89AxUZ0aYdF39BYoNFa324SodPCA=
github.com/alecthomas/repr v0.0.0-20180818092828-117648cd9897 h1:p9Sln00KOTlrYkxI1zYWl1QLnEqAqEARBEYa8FQnQcY=
github.com/alecthomas/repr v0.0.0-20180818092828-117648cd9897/go.mod h1:xTS7Pm1pD1mvyM075QCDSRqH6qRLXylzS24ZTpRiSzQ=
github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
github.com/andybalholm/brotli v0.0.0-20190621154722-5f990b63d2d6 h1:bZ28Hqta7TFAK3Q08CMvv8y3/8ATaEqv2nGoc6yff6c=
@ -143,6 +159,9 @@ github.com/cznic/mathutil v0.0.0-20181122101859-297441e03548 h1:iwZdTE0PVqJCos1v
github.com/cznic/mathutil v0.0.0-20181122101859-297441e03548/go.mod h1:e6NPNENfs9mPDVNRekM7lKScauxd5kXTr1Mfyig6TDM=
github.com/cznic/strutil v0.0.0-20181122101858-275e90344537 h1:MZRmHqDBd0vxNwenEbKSQqRVT24d3C05ft8kduSwlqM=
github.com/cznic/strutil v0.0.0-20181122101858-275e90344537/go.mod h1:AHHPPPXTw0h6pVabbcbyGRK1DckRn7r/STdZEeIDzZc=
github.com/daaku/go.zipexe v1.0.0/go.mod h1:z8IiR6TsVLEYKwXAoE/I+8ys/sDkgTzSL0CLnGVd57E=
github.com/danwakefield/fnmatch v0.0.0-20160403171240-cbb64ac3d964 h1:y5HC9v93H5EPKqaS1UYVg1uYah5Xf51mBfIoWehClUQ=
github.com/danwakefield/fnmatch v0.0.0-20160403171240-cbb64ac3d964/go.mod h1:Xd9hchkHSWYkEqJwUGisez3G1QY8Ryz0sdWrLPMGjLk=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
@ -155,6 +174,9 @@ github.com/denisenkom/go-mssqldb v0.0.0-20200428022330-06a60b6afbbc/go.mod h1:xb
github.com/dgrijalva/jwt-go v3.2.0+incompatible h1:7qlOGliEKZXTDg6OTjfoBKDXWrumCAMpl/TFQ4/5kLM=
github.com/dgrijalva/jwt-go v3.2.0+incompatible/go.mod h1:E3ru+11k8xSBh+hMPgOLZmtrrCbhqsmaPHjLKYnJCaQ=
github.com/dgryski/go-sip13 v0.0.0-20181026042036-e10d5fee7954/go.mod h1:vAd38F8PWV+bWy6jNmig1y/TA+kYO4g3RSRF0IAv0no=
github.com/dlclark/regexp2 v1.1.6/go.mod h1:2pZnwuY/m+8K6iRw6wQdMtk+rH5tNGR1i55kozfMjCc=
github.com/dlclark/regexp2 v1.2.0 h1:8sAhBGEM0dRWogWqWyQeIJnxjWO6oIjl8FKqREDsGfk=
github.com/dlclark/regexp2 v1.2.0/go.mod h1:2pZnwuY/m+8K6iRw6wQdMtk+rH5tNGR1i55kozfMjCc=
github.com/docker/go-units v0.3.3/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
github.com/docker/go-units v0.4.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
github.com/dsnet/compress v0.0.1 h1:PlZu0n3Tuv04TzpfPbrnI0HW/YwodEXDS+oPKahKF0Q=
@ -348,12 +370,16 @@ github.com/gopherjs/gopherjs v0.0.0-20190910122728-9d188e94fb99 h1:twflg0XRTjwKp
github.com/gopherjs/gopherjs v0.0.0-20190910122728-9d188e94fb99/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY=
github.com/gorilla/context v1.1.1 h1:AWwleXJkX/nhcU9bZSnZoi3h/qGYqQAGhq6zZe/aQW8=
github.com/gorilla/context v1.1.1/go.mod h1:kBGZzfjB9CEq2AlWe17Uuf7NDRt0dE0s8S51q0aT7Yg=
github.com/gorilla/csrf v1.6.0/go.mod h1:7tSf8kmjNYr7IWDCYhd3U8Ck34iQ/Yw5CJu7bAkHEGI=
github.com/gorilla/css v1.0.0 h1:BQqNyPTi50JCFMTw/b67hByjMVXZRwGha6wxVGkeihY=
github.com/gorilla/css v1.0.0/go.mod h1:Dn721qIggHpt4+EFCcTLTU/vk5ySda2ReITrtgBl60c=
github.com/gorilla/handlers v1.4.1/go.mod h1:Qkdc/uu4tH4g6mTK6auzZ766c4CA0Ng8+o/OAirnOIQ=
github.com/gorilla/handlers v1.4.2 h1:0QniY0USkHQ1RGCLfKxeNHK9bkDHGRYGNDFBCS+YARg=
github.com/gorilla/handlers v1.4.2/go.mod h1:Qkdc/uu4tH4g6mTK6auzZ766c4CA0Ng8+o/OAirnOIQ=
github.com/gorilla/mux v1.6.2 h1:Pgr17XVTNXAk3q/r4CpKzC5xBM/qW1uVLV+IhRZpIIk=
github.com/gorilla/mux v1.6.2/go.mod h1:1lud6UwP+6orDFRuTfBEV8e9/aOM/c4fVVCaMa2zaAs=
github.com/gorilla/mux v1.7.3 h1:gnP5JzjVOuiZD07fKKToCAOjS0yOpj/qPETTXCCS6hw=
github.com/gorilla/mux v1.7.3/go.mod h1:1lud6UwP+6orDFRuTfBEV8e9/aOM/c4fVVCaMa2zaAs=
github.com/gorilla/pat v0.0.0-20180118222023-199c85a7f6d1 h1:LqbZZ9sNMWVjeXS4NN5oVvhMjDyLhmA1LG86oSo+IqY=
github.com/gorilla/pat v0.0.0-20180118222023-199c85a7f6d1/go.mod h1:YeAe0gNeiNT5hoiZRI4yiOky6jVdNvfO2N6Kav/HmxY=
github.com/gorilla/securecookie v1.1.1 h1:miw7JPhV+b/lAHSXz4qd/nN9jRiAFV5FwjeKyCS8BvQ=
@ -463,11 +489,17 @@ github.com/mailru/easyjson v0.7.0/go.mod h1:KAzv3t3aY1NaHWoQz1+4F1ccyAH66Jk7yos7
github.com/markbates/going v1.0.0/go.mod h1:I6mnB4BPnEeqo85ynXIx1ZFLLbtiLHNXVgWeFO9OGOA=
github.com/markbates/goth v1.61.2 h1:jDowrUH5qw8KGuQdKwFhLzkXkTYCIPfz3LHADJsiPIs=
github.com/markbates/goth v1.61.2/go.mod h1:qh2QfwZoWRucQ+DR5KVKC6dUGkNCToWh4vS45GIzFsY=
github.com/mattn/go-colorable v0.0.9/go.mod h1:9vuHe8Xs5qXnSaW/c/ABM9alt+Vo+STaOChaDxuIBZU=
github.com/mattn/go-colorable v0.1.4 h1:snbPLB8fVfU9iwbbo30TPtbLRzwWu6aJS6Xh4eaaviA=
github.com/mattn/go-colorable v0.1.4/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE=
github.com/mattn/go-colorable v0.1.6 h1:6Su7aK7lXmJ/U79bYtBjLNaha4Fs1Rg9plHpcH+vvnE=
github.com/mattn/go-colorable v0.1.6/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc=
github.com/mattn/go-isatty v0.0.4/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4=
github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
github.com/mattn/go-isatty v0.0.11 h1:FxPOTFNqGkuDUGi3H/qkUbQO4ZiBa2brKq5r0l8TGeM=
github.com/mattn/go-isatty v0.0.11/go.mod h1:PhnuNfih5lzO57/f3n+odYbM4JtupLOxQOAqxQCu2WE=
github.com/mattn/go-isatty v0.0.12 h1:wuysRhFDzyxgEmMf5xjvJ2M9dZoWAXNNr5LSBS7uHXY=
github.com/mattn/go-isatty v0.0.12/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Kysco4FUpU=
github.com/mattn/go-runewidth v0.0.7 h1:Ei8KR0497xHyKJPAv59M1dkC+rOZCMBJ+t3fZ+twI54=
github.com/mattn/go-runewidth v0.0.7/go.mod h1:H031xJmbD/WCDINGzjvQ9THkh0rPKHF+m2gUSrubnMI=
github.com/mattn/go-sqlite3 v1.10.0/go.mod h1:FPy6KqzDD04eiIsT53CuJW3U88zkxoIYsOqkbpncsNc=
@ -510,6 +542,7 @@ github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e h1:fD57ERR4JtEqsWb
github.com/niemeyer/pretty v0.0.0-20200227124842-a10e7caefd8e/go.mod h1:zD1mROLANZcx1PVRCS0qkT7pwLkGfwJo4zjcN/Tysno=
github.com/niklasfasching/go-org v0.1.9 h1:Toz8WMIt+qJb52uYEk1YD/muLuOOmRt1CfkV+bKVMkI=
github.com/niklasfasching/go-org v0.1.9/go.mod h1:AsLD6X7djzRIz4/RFZu8vwRL0VGjUvGZCCH1Nz0VdrU=
github.com/nkovacs/streamquote v0.0.0-20170412213628-49af9bddb229/go.mod h1:0aYXnNPJ8l7uZxf45rWW1a/uME32OF0rhiYGNQ2oF2E=
github.com/nwaples/rardecode v1.0.0 h1:r7vGuS5akxOnR4JQSkko62RJ1ReCMXxQRPtxsiFMBOs=
github.com/nwaples/rardecode v1.0.0/go.mod h1:5DzqNKiOdpKKBH87u8VlvAnPZMXcGRhxWkRpHbbfGS0=
github.com/oklog/ulid v1.3.1/go.mod h1:CirwcVhetQ6Lv90oh/F+FBtV6XMibvdAFo93nm5qn4U=
@ -580,6 +613,7 @@ github.com/remyoudompheng/bigfft v0.0.0-20190321074620-2f0d2b0e0001/go.mod h1:qq
github.com/rogpeppe/fastuuid v0.0.0-20150106093220-6724a57986af/go.mod h1:XWv6SoW27p1b0cqNHllgS5HIMJraePCO15w5zCzIWYg=
github.com/russross/blackfriday v1.5.2 h1:HyvC0ARfnZBqnXwABFeSZHpKvJHJJfPz81GNueLj0oo=
github.com/russross/blackfriday v1.5.2/go.mod h1:JO/DiYxRf+HjHt06OyowR9PTA263kcR/rfWxYHBV53g=
github.com/sergi/go-diff v1.0.0/go.mod h1:0CfEIISq7TuYL3j771MWULgwwjU+GofnZX9QAmXWZgo=
github.com/sergi/go-diff v1.1.0 h1:we8PVUC3FE2uYfodKH/nBHMSetSfHDR6scGdBi+erh0=
github.com/sergi/go-diff v1.1.0/go.mod h1:STckp+ISIX8hZLjrqAeVduY0gWCT9IjLuqbuNXdaHfM=
github.com/shurcooL/httpfs v0.0.0-20190527155220-6a4d4a70508b h1:4kg1wyftSKxLtnPAvcRWakIPpokB9w780/KwrNLnfPA=
@ -658,6 +692,8 @@ github.com/unknwon/paginater v0.0.0-20151104151617-7748a72e0141 h1:Z79lyIznnziKA
github.com/unknwon/paginater v0.0.0-20151104151617-7748a72e0141/go.mod h1:TBwoao3Q4Eb/cp+dHbXDfRTrZSsj/k7kLr2j1oWRWC0=
github.com/urfave/cli v1.20.0 h1:fDqGv3UG/4jbVl/QkFwEdddtEDjh/5Ov6X+0B/3bPaw=
github.com/urfave/cli v1.20.0/go.mod h1:70zkFmudgCuE/ngEzBv17Jvp/497gISqfk5gWijbERA=
github.com/valyala/bytebufferpool v1.0.0/go.mod h1:6bBcMArwyJ5K/AmCkWv1jt77kVWyCJ6HpOuEn7z0Csc=
github.com/valyala/fasttemplate v1.0.1/go.mod h1:UQGH1tvbgY+Nz5t2n7tXsz52dQxojPUpymEIMZ47gx8=
github.com/willf/bitset v1.1.10 h1:NotGKqX0KwQ72NUzqrjZq5ipPNDQex9lo3WpaS8L2sc=
github.com/willf/bitset v1.1.10/go.mod h1:RjeCKbqT1RxIR/KWY6phxZiaY1IyutSBfGjNPySAYV4=
github.com/xanzy/go-gitlab v0.31.0 h1:+nHztQuCXGSMluKe5Q9IRaPdz6tO8O0gMkQ0vqGpiBk=
@ -671,8 +707,11 @@ github.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77/go.mod h1:
github.com/yohcop/openid-go v1.0.0 h1:EciJ7ZLETHR3wOtxBvKXx9RV6eyHZpCaSZ1inbBaUXE=
github.com/yohcop/openid-go v1.0.0/go.mod h1:/408xiwkeItSPJZSTPF7+VtZxPkPrRRpRNK2vjGh6yI=
github.com/yuin/goldmark v1.1.7/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.1.22/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark v1.1.25 h1:isv+Q6HQAmmL2Ofcmg8QauBmDPlUUnSoNhEcC940Rds=
github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark-highlighting v0.0.0-20200307114337-60d527fdb691 h1:VWSxtAiQNh3zgHJpdpkpVYjTPqRE3P6UZCOPa1nRDio=
github.com/yuin/goldmark-highlighting v0.0.0-20200307114337-60d527fdb691/go.mod h1:YLF3kDffRfUH/bTxOxHhV6lxwIB3Vfj91rEwNMS9MXo=
github.com/yuin/goldmark v1.1.32 h1:5tjfNdR2ki3yYQ842+eX2sQHeiwpKJ0RnHO4IYOc4V8=
github.com/yuin/goldmark v1.1.32/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=
github.com/yuin/goldmark-meta v0.0.0-20191126180153-f0638e958b60 h1:gZucqLjL1eDzVWrXj4uiWeMbAopJlBR2mKQAsTGdPwo=
@ -772,6 +811,7 @@ golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5h
golang.org/x/sys v0.0.0-20181107165924-66b7b1311ac8/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20181122145206-62eef0e2fa9b/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20181128092732-4ed8d59d0b35/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20181205085412-a5c9d58dba9a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20181221143128-b4a75ba826a6/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
@ -792,11 +832,14 @@ golang.org/x/sys v0.0.0-20190907184412-d223b2b6db03/go.mod h1:h1NjWce9XRLGQEsW7w
golang.org/x/sys v0.0.0-20191010194322-b09406accb47 h1:/XfQ9z7ib8eEJX2hdgFTZJ/ntt0swNk5oYBziWeTCvY=
golang.org/x/sys v0.0.0-20191010194322-b09406accb47/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200116001909-b77594299b42/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200202164722-d101bd2416d5/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527 h1:uYVVQ9WP/Ds2ROhcaGPeIdVq0RIXVLwsHlnvJ+cT1So=
golang.org/x/sys v0.0.0-20200302150141-5c8b2ff67527/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd h1:xhmwyvizuTgC2qz7ZlMluP20uW+C3Rm0FD/WLDX8884=
golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200413165638-669c56c373c4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200615200032-f1bc736245b1 h1:ogLJMz+qpzav7lGMh10LMvAkM/fAoGlaiiHYiFYdm80=
golang.org/x/sys v0.0.0-20200615200032-f1bc736245b1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=

@ -1,151 +1,148 @@
// Copyright 2015 The Gogs Authors. All rights reserved.
// Copyright 2020 The Gitea Authors. All rights reserved.
// Use of this source code is governed by a MIT-style
// license that can be found in the LICENSE file.
package highlight
import (
"path"
"bufio"
"bytes"
"path/filepath"
"strings"
"sync"
"code.gitea.io/gitea/modules/log"
"code.gitea.io/gitea/modules/setting"
"github.com/alecthomas/chroma/formatters/html"
"github.com/alecthomas/chroma/lexers"
"github.com/alecthomas/chroma/styles"
)
// don't index files larger than this many bytes for performance purposes
const sizeLimit = 1000000
var (
// File name should ignore highlight.
ignoreFileNames = map[string]bool{
"license": true,
"copying": true,
}
// For custom user mapping
highlightMapping = map[string]string{}
once sync.Once
)
// NewContext loads custom highlight map from local config
func NewContext() {
once.Do(func() {
keys := setting.Cfg.Section("highlight.mapping").Keys()
for i := range keys {
highlightMapping[keys[i].Name()] = keys[i].Value()
}
})
}
// Code returns a HTML version of code string with chroma syntax highlighting classes
func Code(fileName, code string) string {
NewContext()
// File names that are representing highlight classes.
highlightFileNames = map[string]string{
"dockerfile": "dockerfile",
"makefile": "makefile",
"gnumakefile": "makefile",
"cmakelists.txt": "cmake",
if len(code) > sizeLimit {
return code
}
formatter := html.New(html.WithClasses(true),
html.WithLineNumbers(false),
html.PreventSurroundingPre(true),
)
if formatter == nil {
log.Error("Couldn't create chroma formatter")
return code
}
// Extensions that are same as highlight classes.
// See hljs.listLanguages() for list of language names.
highlightExts = map[string]struct{}{
".applescript": {},
".arm": {},
".as": {},
".bash": {},
".bat": {},
".c": {},
".cmake": {},
".cpp": {},
".cs": {},
".css": {},
".dart": {},
".diff": {},
".django": {},
".go": {},
".gradle": {},
".groovy": {},
".haml": {},
".handlebars": {},
".html": {},
".ini": {},
".java": {},
".json": {},
".less": {},
".lua": {},
".php": {},
".scala": {},
".scss": {},
".sql": {},
".swift": {},
".ts": {},
".xml": {},
".yaml": {},
htmlbuf := bytes.Buffer{}
htmlw := bufio.NewWriter(&htmlbuf)
if val, ok := highlightMapping[filepath.Ext(fileName)]; ok {
//change file name to one with mapped extension so we look that up instead
fileName = "mapped." + val
}
// Extensions that are not same as highlight classes.
highlightMapping = map[string]string{
".ahk": "autohotkey",
".crmsh": "crmsh",
".dash": "shell",
".erl": "erlang",
".escript": "erlang",
".ex": "elixir",
".exs": "elixir",
".f": "fortran",
".f77": "fortran",
".f90": "fortran",
".f95": "fortran",
".feature": "gherkin",
".fish": "shell",
".for": "fortran",
".hbs": "handlebars",
".hs": "haskell",
".hx": "haxe",
".js": "javascript",
".jsx": "javascript",
".ksh": "shell",
".kt": "kotlin",
".l": "ocaml",
".ls": "livescript",
".md": "markdown",
".mjs": "javascript",
".mli": "ocaml",
".mll": "ocaml",
".mly": "ocaml",
".patch": "diff",
".pl": "perl",
".pm": "perl",
".ps1": "powershell",
".psd1": "powershell",
".psm1": "powershell",
".py": "python",
".pyw": "python",
".rb": "ruby",
".rs": "rust",
".scpt": "applescript",
".scptd": "applescript",
".sh": "bash",
".tcsh": "shell",
".ts": "typescript",
".tsx": "typescript",
".txt": "plaintext",
".vb": "vbnet",
".vbs": "vbscript",
".yml": "yaml",
".zsh": "shell",
lexer := lexers.Match(fileName)
if lexer == nil {
lexer = lexers.Fallback
}
)
// NewContext loads highlight map
func NewContext() {
keys := setting.Cfg.Section("highlight.mapping").Keys()
for i := range keys {
highlightMapping[keys[i].Name()] = keys[i].Value()
iterator, err := lexer.Tokenise(nil, string(code))
if err != nil {
log.Error("Can't tokenize code: %v", err)
return code
}
// style not used for live site but need to pass something
err = formatter.Format(htmlw, styles.GitHub, iterator)
if err != nil {
log.Error("Can't format code: %v", err)
return code
}
htmlw.Flush()
return htmlbuf.String()
}
// FileNameToHighlightClass returns the best match for highlight class name
// based on the rule of highlight.js.
func FileNameToHighlightClass(fname string) string {
fname = strings.ToLower(fname)
if ignoreFileNames[fname] {
return "nohighlight"
// File returns map with line lumbers and HTML version of code with chroma syntax highlighting classes
func File(numLines int, fileName string, code []byte) map[int]string {
NewContext()
if len(code) > sizeLimit {
return plainText(string(code), numLines)
}
formatter := html.New(html.WithClasses(true),
html.WithLineNumbers(false),
html.PreventSurroundingPre(true),
)
if formatter == nil {
log.Error("Couldn't create chroma formatter")
return plainText(string(code), numLines)
}
htmlbuf := bytes.Buffer{}
htmlw := bufio.NewWriter(&htmlbuf)
if val, ok := highlightMapping[filepath.Ext(fileName)]; ok {
fileName = "test." + val
}
lexer := lexers.Match(fileName)
if lexer == nil {
lexer = lexers.Analyse(string(code))
if lexer == nil {
lexer = lexers.Fallback
}
}
if name, ok := highlightFileNames[fname]; ok {
return name
iterator, err := lexer.Tokenise(nil, string(code))
if err != nil {
log.Error("Can't tokenize code: %v", err)
return plainText(string(code), numLines)
}
ext := path.Ext(fname)
if _, ok := highlightExts[ext]; ok {
return ext[1:]
err = formatter.Format(htmlw, styles.GitHub, iterator)
if err != nil {
log.Error("Can't format code: %v", err)
return plainText(string(code), numLines)
}
name, ok := highlightMapping[ext]
if ok {
return name
htmlw.Flush()
m := make(map[int]string, numLines)
for k, v := range strings.SplitN(htmlbuf.String(), "\n", numLines) {
line := k + 1
m[line] = string(v)
}
return m
}
return ""
// return unhiglighted map
func plainText(code string, numLines int) map[int]string {
m := make(map[int]string, numLines)
for k, v := range strings.SplitN(string(code), "\n", numLines) {
line := k + 1
m[line] = string(v)
}
return m
}

@ -6,8 +6,6 @@ package code
import (
"bytes"
"html"
gotemplate "html/template"
"strings"
"code.gitea.io/gitea/modules/highlight"
@ -23,9 +21,8 @@ type Result struct {
UpdatedUnix timeutil.TimeStamp
Language string
Color string
HighlightClass string
LineNumbers []int
FormattedLines gotemplate.HTML
FormattedLines string
}
func indices(content string, selectionStartIndex, selectionEndIndex int) (int, int) {
@ -80,19 +77,13 @@ func searchResult(result *SearchResult, startIndex, endIndex int) (*Result, erro
openActiveIndex := util.Max(result.StartIndex-index, 0)
closeActiveIndex := util.Min(result.EndIndex-index, len(line))
err = writeStrings(&formattedLinesBuffer,
`<li>`,
html.EscapeString(line[:openActiveIndex]),
`<span class='active'>`,
html.EscapeString(line[openActiveIndex:closeActiveIndex]),
`</span>`,
html.EscapeString(line[closeActiveIndex:]),
`</li>`,
line[:openActiveIndex],
line[openActiveIndex:closeActiveIndex],
line[closeActiveIndex:],
)
} else {
err = writeStrings(&formattedLinesBuffer,
`<li>`,
html.EscapeString(line),
`</li>`,
line,
)
}
if err != nil {
@ -109,9 +100,8 @@ func searchResult(result *SearchResult, startIndex, endIndex int) (*Result, erro
UpdatedUnix: result.UpdatedUnix,
Language: result.Language,
Color: result.Color,
HighlightClass: highlight.FileNameToHighlightClass(result.Filename),
LineNumbers: lineNumbers,
FormattedLines: gotemplate.HTML(formattedLinesBuffer.String()),
FormattedLines: highlight.Code(result.Filename, formattedLinesBuffer.String()),
}, nil
}

@ -15,7 +15,9 @@ import (
"code.gitea.io/gitea/modules/setting"
giteautil "code.gitea.io/gitea/modules/util"
chromahtml "github.com/alecthomas/chroma/formatters/html"
"github.com/yuin/goldmark"
"github.com/yuin/goldmark-highlighting"
meta "github.com/yuin/goldmark-meta"
"github.com/yuin/goldmark/extension"
"github.com/yuin/goldmark/parser"
@ -49,6 +51,30 @@ func render(body []byte, urlPrefix string, metas map[string]string, wikiMarkdown
extension.TaskList,
extension.DefinitionList,
common.FootnoteExtension,
highlighting.NewHighlighting(
highlighting.WithFormatOptions(
chromahtml.WithClasses(true),
chromahtml.PreventSurroundingPre(true),
),
highlighting.WithWrapperRenderer(func(w util.BufWriter, c highlighting.CodeBlockContext, entering bool) {
language, _ := c.Language()
if language == nil {
language = []byte("text")
}
if entering {
// include language-x class as part of commonmark spec
_, err := w.WriteString("<pre><code class=\"chroma language-" + string(language) + "\">")
if err != nil {
return
}
} else {
_, err := w.WriteString("</code></pre>")
if err != nil {
return
}
}
}),
),
meta.Meta,
),
goldmark.WithParserOptions(

@ -37,8 +37,8 @@ func NewSanitizer() {
// ReplaceSanitizer replaces the current sanitizer to account for changes in settings
func ReplaceSanitizer() {
sanitizer.policy = bluemonday.UGCPolicy()
// We only want to allow HighlightJS specific classes for code blocks
sanitizer.policy.AllowAttrs("class").Matching(regexp.MustCompile(`^language-[\w-]+$`)).OnElements("code")
// For Chroma markdown plugin
sanitizer.policy.AllowAttrs("class").Matching(regexp.MustCompile(`^(chroma )?language-[\w-]+$`)).OnElements("code")
// Checkboxes
sanitizer.policy.AllowAttrs("type").Matching(regexp.MustCompile(`^checkbox$`)).OnElements("input")
@ -65,8 +65,8 @@ func ReplaceSanitizer() {
// Allow classes for emojis
sanitizer.policy.AllowAttrs("class").Matching(regexp.MustCompile(`emoji`)).OnElements("img")
// Allow icons, checkboxes and emojis on span
sanitizer.policy.AllowAttrs("class").Matching(regexp.MustCompile(`^((icon(\s+[\p{L}\p{N}_-]+)+)|(ui checkbox)|(ui checked checkbox)|(emoji))$`)).OnElements("span")
// Allow icons, checkboxes, emojis, and chroma syntax on span
sanitizer.policy.AllowAttrs("class").Matching(regexp.MustCompile(`^((icon(\s+[\p{L}\p{N}_-]+)+)|(ui checkbox)|(ui checked checkbox)|(emoji))$|^([a-z][a-z0-9]{0,2})$`)).OnElements("span")
// Allow generally safe attributes
generalSafeAttrs := []string{"abbr", "accept", "accept-charset",

@ -47,7 +47,8 @@ func TestGetDiffPreview(t *testing.T) {
IsSubmodule: false,
Sections: []*gitdiff.DiffSection{
{
Name: "",
FileName: "README.md",
Name: "",
Lines: []*gitdiff.DiffLine{
{
LeftIdx: 0,

10
package-lock.json generated

@ -3829,11 +3829,6 @@
"domelementtype": "1"
}
},
"domino": {
"version": "2.1.5",
"resolved": "https://registry.npmjs.org/domino/-/domino-2.1.5.tgz",
"integrity": "sha512-vMDo7f6ogUV9PkzmxXLiXzJkJZqU09Le4C40mj+HmAGS/2FPmdetoNOQZXpu2kekn0GJKvtwKMAVoruTj60Xww=="
},
"dompurify": {
"version": "2.0.11",
"resolved": "https://registry.npmjs.org/dompurify/-/dompurify-2.0.11.tgz",
@ -6912,11 +6907,6 @@
"resolved": "https://registry.npmjs.org/hex-color-regex/-/hex-color-regex-1.1.0.tgz",
"integrity": "sha512-l9sfDFsuqtOqKDsQdqrMRk0U85RZc0RtOR9yPI7mRVOa4FsR/BVnZ0shmQRM96Ji99kYZP/7hn1cedc1+ApsTQ=="
},
"highlight.js": {
"version": "10.0.3",
"resolved": "https://registry.npmjs.org/highlight.js/-/highlight.js-10.0.3.tgz",
"integrity": "sha512-9FG7SSzv9yOY5CGGxfI6NDm7xLYtMOjKtPBxw7Zff3t5UcRcUNTGEeS8lNjhceL34KeetLMoGMFTGoaa83HwyQ=="
},
"hmac-drbg": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/hmac-drbg/-/hmac-drbg-1.0.1.tgz",

@ -16,12 +16,10 @@
"core-js": "3.6.5",
"css-loader": "3.5.3",
"cssnano-webpack-plugin": "1.0.3",
"domino": "2.1.5",
"dropzone": "5.7.1",
"fast-glob": "3.2.2",
"file-loader": "6.0.0",
"fomantic-ui": "2.8.6",
"highlight.js": "10.0.3",
"jquery": "3.5.1",
"jquery.are-you-sure": "1.9.0",
"less-loader": "6.1.0",

@ -19,7 +19,6 @@ import (
"code.gitea.io/gitea/modules/git"
"code.gitea.io/gitea/modules/highlight"
"code.gitea.io/gitea/modules/log"
"code.gitea.io/gitea/modules/markup"
"code.gitea.io/gitea/modules/setting"
"code.gitea.io/gitea/modules/timeutil"
)
@ -109,10 +108,7 @@ func RefBlame(ctx *context.Context) {
ctx.Data["TreeLink"] = treeLink
ctx.Data["TreeNames"] = treeNames
ctx.Data["BranchLink"] = branchLink
ctx.Data["HighlightClass"] = highlight.FileNameToHighlightClass(entry.Name())
if !markup.IsReadmeFile(blob.Name()) {
ctx.Data["RequireHighlightJS"] = true
}
ctx.Data["RawFileLink"] = rawLink + "/" + ctx.Repo.TreePath
ctx.Data["PageIsViewCode"] = true
@ -236,11 +232,11 @@ func renderBlame(ctx *context.Context, blameParts []git.BlamePart, commitNames m
lineNumbers.WriteString(fmt.Sprintf(`<span id="L%d" data-line-number="%d"></span>`, i, i))
}
//Code line
line = gotemplate.HTMLEscapeString(line)
if i != len(lines)-1 {
line += "\n"
}
fileName := fmt.Sprintf("%v", ctx.Data["FileName"])
line = highlight.Code(fileName, line)
if len(part.Lines)-1 == index && len(blameParts)-1 != pi {
codeLines.WriteString(fmt.Sprintf(`<li class="L%d bottom-line" rel="L%d">%s</li>`, i, i, line))
} else {

@ -9,14 +9,12 @@ import (
"fmt"
"html"
"path"
"path/filepath"
"strings"
"code.gitea.io/gitea/models"
"code.gitea.io/gitea/modules/base"
"code.gitea.io/gitea/modules/context"
"code.gitea.io/gitea/modules/git"
"code.gitea.io/gitea/modules/highlight"
"code.gitea.io/gitea/modules/log"
"code.gitea.io/gitea/modules/setting"
"code.gitea.io/gitea/services/gitdiff"
@ -576,7 +574,6 @@ func CompareDiff(ctx *context.Context) {
ctx.Data["IsRepoToolbarCommits"] = true
ctx.Data["IsDiffCompare"] = true
ctx.Data["RequireHighlightJS"] = true
ctx.Data["RequireTribute"] = true
ctx.Data["RequireSimpleMDE"] = true
ctx.Data["PullRequestWorkInProgressPrefixes"] = setting.Repository.PullRequest.WorkInProgressPrefixes
@ -657,7 +654,6 @@ func ExcerptBlob(ctx *context.Context) {
}
ctx.Data["section"] = section
ctx.Data["fileName"] = filePath
ctx.Data["highlightClass"] = highlight.FileNameToHighlightClass(filepath.Base(filePath))
ctx.Data["AfterCommitID"] = commitID
ctx.Data["Anchor"] = anchor
ctx.HTML(200, tplBlobExcerpt)

@ -13,6 +13,7 @@ import (
"io/ioutil"
"net/url"
"path"
"strconv"
"strings"
"code.gitea.io/gitea/models"
@ -42,6 +43,15 @@ type namedBlob struct {
blob *git.Blob
}
func linesBytesCount(s []byte) int {
nl := []byte{'\n'}
n := bytes.Count(s, nl)
if len(s) > 0 && !bytes.HasSuffix(s, nl) {
n++
}
return n
}
// FIXME: There has to be a more efficient way of doing this
func getReadmeFileFromPath(commit *git.Commit, treePath string) (*namedBlob, error) {
tree, err := commit.SubTree(treePath)
@ -359,7 +369,6 @@ func renderDirectory(ctx *context.Context, treeLink string) {
func renderFile(ctx *context.Context, entry *git.TreeEntry, treeLink, rawLink string) {
ctx.Data["IsViewFile"] = true
blob := entry.Blob()
dataRc, err := blob.DataAsync()
if err != nil {
@ -374,7 +383,6 @@ func renderFile(ctx *context.Context, entry *git.TreeEntry, treeLink, rawLink st
ctx.Data["FileIsSymlink"] = entry.IsLink()
ctx.Data["FileSize"] = fileSize
ctx.Data["FileName"] = blob.Name()
ctx.Data["HighlightClass"] = highlight.FileNameToHighlightClass(blob.Name())
ctx.Data["RawFileLink"] = rawLink + "/" + ctx.Repo.TreePath
buf := make([]byte, 1024)
@ -453,7 +461,6 @@ func renderFile(ctx *context.Context, entry *git.TreeEntry, treeLink, rawLink st
d, _ := ioutil.ReadAll(dataRc)
buf = charset.ToUTF8WithFallback(append(buf, d...))
readmeExist := markup.IsReadmeFile(blob.Name())
ctx.Data["ReadmeExist"] = readmeExist
if markupType := markup.Type(blob.Name()); markupType != "" {
@ -466,42 +473,11 @@ func renderFile(ctx *context.Context, entry *git.TreeEntry, treeLink, rawLink st
gotemplate.HTMLEscapeString(string(buf)), "\n", `<br>`, -1,
)
} else {
// Building code view blocks with line number on server side.
var fileContent string
if content, err := charset.ToUTF8WithErr(buf); err != nil {
log.Error("ToUTF8WithErr: %v", err)
fileContent = string(buf)
} else {
fileContent = content
}
var output bytes.Buffer
lines := strings.Split(fileContent, "\n")
ctx.Data["NumLines"] = len(lines)
if len(lines) == 1 && lines[0] == "" {
// If the file is completely empty, we show zero lines at the line counter
ctx.Data["NumLines"] = 0
}
buf = charset.ToUTF8WithFallback(buf)
lineNums := linesBytesCount(buf)
ctx.Data["NumLines"] = strconv.Itoa(lineNums)
ctx.Data["NumLinesSet"] = true
//Remove blank line at the end of file
if len(lines) > 0 && lines[len(lines)-1] == "" {
lines = lines[:len(lines)-1]
}
for index, line := range lines {
line = gotemplate.HTMLEscapeString(line)
if index != len(lines)-1 {
line += "\n"
}
output.WriteString(fmt.Sprintf(`<li class="L%d" rel="L%d">%s</li>`, index+1, index+1, line))
}
ctx.Data["FileContent"] = gotemplate.HTML(output.String())
output.Reset()
for i := 0; i < len(lines); i++ {
output.WriteString(fmt.Sprintf(`<span id="L%[1]d" data-line-number="%[1]d"></span>`, i+1))
}
ctx.Data["LineNums"] = gotemplate.HTML(output.String())
ctx.Data["FileContent"] = highlight.File(lineNums, blob.Name(), buf)
}
if !isLFSFile {
if ctx.Repo.CanEnableEditor() {
@ -645,7 +621,6 @@ func renderCode(ctx *context.Context) {
title += ": " + ctx.Repo.Repository.Description
}
ctx.Data["Title"] = title
ctx.Data["RequireHighlightJS"] = true
branchLink := ctx.Repo.RepoLink + "/src/" + ctx.Repo.BranchNameSubURL()
treeLink := branchLink

@ -10,7 +10,6 @@ import (
"bytes"
"context"
"fmt"
"html"
"html/template"
"io"
"io/ioutil"
@ -164,15 +163,16 @@ func getDiffLineSectionInfo(treePath, line string, lastLeftIdx, lastRightIdx int
// escape a line's content or return <br> needed for copy/paste purposes
func getLineContent(content string) string {
if len(content) > 0 {
return html.EscapeString(content)
return content
}
return "<br>"
return "\n"
}
// DiffSection represents a section of a DiffFile.
type DiffSection struct {
Name string
Lines []*DiffLine
FileName string
Name string
Lines []*DiffLine
}
var (
@ -181,24 +181,23 @@ var (
codeTagSuffix = []byte(`</span>`)
)
func diffToHTML(diffs []diffmatchpatch.Diff, lineType DiffLineType) template.HTML {
func diffToHTML(fileName string, diffs []diffmatchpatch.Diff, lineType DiffLineType) template.HTML {
buf := bytes.NewBuffer(nil)
for i := range diffs {
switch {
case diffs[i].Type == diffmatchpatch.DiffInsert && lineType == DiffLineAdd:
buf.Write(addedCodePrefix)
buf.WriteString(getLineContent(diffs[i].Text))
buf.WriteString(highlight.Code(fileName, diffs[i].Text))
buf.Write(codeTagSuffix)
case diffs[i].Type == diffmatchpatch.DiffDelete && lineType == DiffLineDel:
buf.Write(removedCodePrefix)
buf.WriteString(getLineContent(diffs[i].Text))
buf.WriteString(highlight.Code(fileName, diffs[i].Text))
buf.Write(codeTagSuffix)
case diffs[i].Type == diffmatchpatch.DiffEqual:
buf.WriteString(getLineContent(diffs[i].Text))
buf.WriteString(highlight.Code(fileName, getLineContent(diffs[i].Text)))
}
}
return template.HTML(buf.Bytes())
}
@ -256,6 +255,7 @@ func (diffSection *DiffSection) GetComputedInlineDiffFor(diffLine *DiffLine) tem
if setting.Git.DisableDiffHighlight {
return template.HTML(getLineContent(diffLine.Content[1:]))
}
var (
compareDiffLine *DiffLine
diff1 string
@ -264,31 +264,32 @@ func (diffSection *DiffSection) GetComputedInlineDiffFor(diffLine *DiffLine) tem
// try to find equivalent diff line. ignore, otherwise
switch diffLine.Type {
case DiffLineSection:
return template.HTML(getLineContent(diffLine.Content[1:]))
case DiffLineAdd:
compareDiffLine = diffSection.GetLine(DiffLineDel, diffLine.RightIdx)
if compareDiffLine == nil {
return template.HTML(getLineContent(diffLine.Content[1:]))
return template.HTML(highlight.Code(diffSection.FileName, diffLine.Content[1:]+"\n"))
}
diff1 = compareDiffLine.Content
diff2 = diffLine.Content
case DiffLineDel:
compareDiffLine = diffSection.GetLine(DiffLineAdd, diffLine.LeftIdx)
if compareDiffLine == nil {
return template.HTML(getLineContent(diffLine.Content[1:]))
return template.HTML(highlight.Code(diffSection.FileName, diffLine.Content[1:]+"\n"))
}
diff1 = diffLine.Content
diff2 = compareDiffLine.Content
default:
if strings.IndexByte(" +-", diffLine.Content[0]) > -1 {
return template.HTML(getLineContent(diffLine.Content[1:]))
return template.HTML(highlight.Code(diffSection.FileName, diffLine.Content[1:]+"\n"))
}
return template.HTML(getLineContent(diffLine.Content))
return template.HTML(highlight.Code(diffSection.FileName, diffLine.Content))
}
diffRecord := diffMatchPatch.DiffMain(diff1[1:], diff2[1:], true)
diffRecord = diffMatchPatch.DiffCleanupEfficiency(diffRecord)
return diffToHTML(diffRecord, diffLine.Type)
return diffToHTML(diffSection.FileName, diffRecord, diffLine.Type)
}
// DiffFile represents a file diff.
@ -313,11 +314,6 @@ func (diffFile *DiffFile) GetType() int {
return int(diffFile.Type)
}
// GetHighlightClass returns highlight class for a filename.
func (diffFile *DiffFile) GetHighlightClass() string {
return highlight.FileNameToHighlightClass(diffFile.Name)
}
// GetTailSection creates a fake DiffLineSection if the last section is not the end of the file
func (diffFile *DiffFile) GetTailSection(gitRepo *git.Repository, leftCommitID, rightCommitID string) *DiffSection {
if len(diffFile.Sections) == 0 || diffFile.Type != DiffFileChange || diffFile.IsBin || diffFile.IsLFSFile {
@ -348,7 +344,7 @@ func (diffFile *DiffFile) GetTailSection(gitRepo *git.Repository, leftCommitID,
LeftIdx: leftLineCount,
RightIdx: rightLineCount,
}}
tailSection := &DiffSection{Lines: []*DiffLine{tailDiffLine}}
tailSection := &DiffSection{FileName: diffFile.Name, Lines: []*DiffLine{tailDiffLine}}
return tailSection
}
@ -405,8 +401,7 @@ const cmdDiffHead = "diff --git "
// TODO: move this function to gogits/git-module
func ParsePatch(maxLines, maxLineCharacters, maxFiles int, reader io.Reader) (*Diff, error) {
var (
diff = &Diff{Files: make([]*DiffFile, 0)}
diff = &Diff{Files: make([]*DiffFile, 0)}
curFile = &DiffFile{}
curSection = &DiffSection{
Lines: make([]*DiffLine, 0, 10),
@ -481,6 +476,7 @@ func ParsePatch(maxLines, maxLineCharacters, maxFiles int, reader io.Reader) (*D
leftLine++
rightLine++
curSection.Lines = append(curSection.Lines, diffLine)
curSection.FileName = curFile.Name
continue
case line[0] == '@':
curSection = &DiffSection{}
@ -492,6 +488,7 @@ func ParsePatch(maxLines, maxLineCharacters, maxFiles int, reader io.Reader) (*D
SectionInfo: lineSectionInfo,
}
curSection.Lines = append(curSection.Lines, diffLine)
curSection.FileName = curFile.Name
// update line number.
leftLine = lineSectionInfo.LeftIdx
rightLine = lineSectionInfo.RightIdx
@ -502,6 +499,7 @@ func ParsePatch(maxLines, maxLineCharacters, maxFiles int, reader io.Reader) (*D
diffLine := &DiffLine{Type: DiffLineAdd, Content: line, RightIdx: rightLine}
rightLine++
curSection.Lines = append(curSection.Lines, diffLine)
curSection.FileName = curFile.Name
continue
case line[0] == '-':
curFile.Deletion++
@ -511,6 +509,7 @@ func ParsePatch(maxLines, maxLineCharacters, maxFiles int, reader io.Reader) (*D
leftLine++
}
curSection.Lines = append(curSection.Lines, diffLine)
curSection.FileName = curFile.Name
case strings.HasPrefix(line, "Binary"):
curFile.IsBin = true
continue

@ -15,6 +15,8 @@ import (
"code.gitea.io/gitea/modules/git"
"code.gitea.io/gitea/modules/setting"
"gopkg.in/ini.v1"
dmp "github.com/sergi/go-diff/diffmatchpatch"
"github.com/stretchr/testify/assert"
)
@ -26,14 +28,15 @@ func assertEqual(t *testing.T, s1 string, s2 template.HTML) {
}
func TestDiffToHTML(t *testing.T) {
assertEqual(t, "foo <span class=\"added-code\">bar</span> biz", diffToHTML([]dmp.Diff{
setting.Cfg = ini.Empty()
assertEqual(t, "foo <span class=\"added-code\">bar</span> biz", diffToHTML("", []dmp.Diff{
{Type: dmp.DiffEqual, Text: "foo "},
{Type: dmp.DiffInsert, Text: "bar"},
{Type: dmp.DiffDelete, Text: " baz"},
{Type: dmp.DiffEqual, Text: " biz"},
}, DiffLineAdd))
assertEqual(t, "foo <span class=\"removed-code\">bar</span> biz", diffToHTML([]dmp.Diff{
assertEqual(t, "foo <span class=\"removed-code\">bar</span> biz", diffToHTML("", []dmp.Diff{
{Type: dmp.DiffEqual, Text: "foo "},
{Type: dmp.DiffDelete, Text: "bar"},
{Type: dmp.DiffInsert, Text: " baz"},

@ -43,7 +43,7 @@
<a href="{{EscapePound $repo.HTMLURL}}/src/commit/{{$result.CommitID}}/{{EscapePound $result.Filename}}#L{{.}}"><span>{{.}}</span></a>
{{end}}
</td>
<td class="lines-code"><pre><code class="{{.HighlightClass}}"><ol class="linenums">{{.FormattedLines}}</ol></code></pre></td>
<td class="lines-code"><pre><code class="chroma"><ol class="linenums">{{.FormattedLines | Safe}}</ol></code></pre></td>
</tr>
</tbody>
</table>

@ -28,7 +28,7 @@
<tr>
<td class="lines-commit">{{.BlameCommitInfo}}</td>
<td class="lines-num">{{.BlameLineNums}}</td>
<td class="lines-code"><pre><code class="{{.HighlightClass}}"><ol class="linenums">{{.BlameContent}}</ol></code></pre></td>
<td class="lines-code"><pre><code class="chroma"><ol class="linenums">{{.BlameContent}}</ol></code></pre></td>
</tr>
</tbody>
</table>

@ -13,14 +13,14 @@
<span class="ui blob-excerpt" data-url="{{$.RepoLink}}/blob_excerpt/{{$.AfterCommitID}}" data-query="{{$line.GetBlobExcerptQuery}}&style=split&direction=" data-anchor="{{$.Anchor}}">{{svg "octicon-fold" 16}}</span>
{{end}}
</td>
<td colspan="5" class="lines-code lines-code-old "><span class="mono wrap{{if $.highlightClass}} language-{{$.highlightClass}}{{else}} nohighlight{{end}}">{{$.section.GetComputedInlineDiffFor $line}}</span></td>
<td colspan="5" class="lines-code lines-code-old "><span class="mono wrap">{{$.section.GetComputedInlineDiffFor $line}}</span></td>
{{else}}
<td class="lines-num lines-num-old" data-line-num="{{if $line.LeftIdx}}{{$line.LeftIdx}}{{end}}"><span rel="{{if $line.LeftIdx}}diff-{{Sha1 $.fileName}}L{{$line.LeftIdx}}{{end}}"></span></td>
<td class="blob-excerpt lines-type-marker lines-type-marker-old">{{if $line.LeftIdx}}<span class="mono" data-type-marker=""></span>{{end}}</td>
<td class="blob-excerpt lines-code lines-code-old halfwidth"><span class="mono wrap{{if $.highlightClass}} language-{{$.highlightClass}}{{else}} nohighlight{{end}}">{{if $line.LeftIdx}}{{$.section.GetComputedInlineDiffFor $line}}{{end}}</span></td>
<td class="blob-excerpt lines-code lines-code-old halfwidth"><span class="mono wrap">{{if $line.LeftIdx}}{{$.section.GetComputedInlineDiffFor $line}}{{end}}</span></td>
<td class="lines-num lines-num-new" data-line-num="{{if $line.RightIdx}}{{$line.RightIdx}}{{end}}"><span rel="{{if $line.RightIdx}}diff-{{Sha1 $.fileName}}R{{$line.RightIdx}}{{end}}"></span></td>
<td class="blob-excerpt lines-type-marker lines-type-marker-new">{{if $line.RightIdx}}<span class="mono" data-type-marker=""></span>{{end}}</td>
<td class="blob-excerpt lines-code lines-code-new halfwidth"><span class="mono wrap{{if $.highlightClass}} language-{{$.highlightClass}}{{else}} nohighlight{{end}}">{{if $line.RightIdx}}{{$.section.GetComputedInlineDiffFor $line}}{{end}}</span></td>
<td class="blob-excerpt lines-code lines-code-new halfwidth"><span class="mono wrap">{{if $line.RightIdx}}{{$.section.GetComputedInlineDiffFor $line}}{{end}}</span></td>
{{end}}
</tr>
{{end}}
@ -44,7 +44,7 @@
<td class="lines-num lines-num-new" data-line-num="{{if $line.RightIdx}}{{$line.RightIdx}}{{end}}"><span rel="{{if $line.RightIdx}}diff-{{Sha1 $.fileName}}R{{$line.RightIdx}}{{end}}"></span></td>
{{end}}
<td class="blob-excerpt lines-type-marker"><span class="mono" data-type-marker="{{$line.GetLineTypeMarker}}"></span></td>
<td class="blob-excerpt lines-code{{if (not $line.RightIdx)}} lines-code-old{{end}}"><span class="mono wrap{{if $.highlightClass}} language-{{$.highlightClass}}{{else}} nohighlight{{end}}">{{$.section.GetComputedInlineDiffFor $line}}</span></td>
<td class="blob-excerpt lines-code{{if (not $line.RightIdx)}} lines-code-old{{end}}"><span class="mono wrap">{{$.section.GetComputedInlineDiffFor $line}}</span></td>
</tr>
{{end}}
{{end}}

@ -114,13 +114,12 @@
<div class="diff-file-body ui attached unstackable table segment">
{{if ne $file.Type 4}}
<div class="file-body file-code code-view has-context-menu code-diff {{if $.IsSplitStyle}}code-diff-split{{else}}code-diff-unified{{end}}">
<table>
<table class="chroma">
<tbody>
{{if $isImage}}
{{template "repo/diff/image_diff" dict "file" . "root" $}}
{{else}}
{{if $.IsSplitStyle}}
{{$highlightClass := $file.GetHighlightClass}}
{{range $j, $section := $file.Sections}}
{{range $k, $line := $section.Lines}}
<tr class="{{DiffLineTypeToStr .GetType}}-code nl-{{$k}} ol-{{$k}}">
@ -136,14 +135,14 @@
<span class="ui blob-excerpt" data-url="{{$.RepoLink}}/blob_excerpt/{{$.AfterCommitID}}" data-query="{{$line.GetBlobExcerptQuery}}&style=split&direction=" data-anchor="diff-{{Sha1 $file.Name}}K{{$line.SectionInfo.RightIdx}}">{{svg "octicon-fold" 16}}</span>
{{end}}
</td>
<td colspan="5" class="lines-code lines-code-old "><span class="mono wrap{{if $highlightClass}} language-{{$highlightClass}}{{else}} nohighlight{{end}}">{{$section.GetComputedInlineDiffFor $line}}</span></td>
<td colspan="5" class="lines-code lines-code-old "><span class="mono wrap">{{$section.GetComputedInlineDiffFor $line}}</span></td>
{{else}}
<td class="lines-num lines-num-old" data-line-num="{{if $line.LeftIdx}}{{$line.LeftIdx}}{{end}}"><span rel="{{if $line.LeftIdx}}diff-{{Sha1 $file.Name}}L{{$line.LeftIdx}}{{end}}"></span></td>
<td class="lines-type-marker lines-type-marker-old">{{if $line.LeftIdx}}<span class="mono" data-type-marker="{{$line.GetLineTypeMarker}}"></span>{{end}}</td>
<td class="lines-code lines-code-old halfwidth">{{if and $.SignedUserID $line.CanComment $.PageIsPullFiles (not (eq .GetType 2))}}<a class="ui green button add-code-comment add-code-comment-left" data-path="{{$file.Name}}" data-side="left" data-idx="{{$line.LeftIdx}}">+</a>{{end}}<span class="mono wrap{{if $highlightClass}} language-{{$highlightClass}}{{else}} nohighlight{{end}}">{{if $line.LeftIdx}}{{$section.GetComputedInlineDiffFor $line}}{{end}}</span></td>
<td class="lines-code lines-code-old halfwidth">{{if and $.SignedUserID $line.CanComment $.PageIsPullFiles (not (eq .GetType 2))}}<a class="ui green button add-code-comment add-code-comment-left" data-path="{{$file.Name}}" data-side="left" data-idx="{{$line.LeftIdx}}">+</a>{{end}}<span class="mono wrap">{{if $line.LeftIdx}}{{$section.GetComputedInlineDiffFor $line}}{{end}}</span></td>
<td class="lines-num lines-num-new" data-line-num="{{if $line.RightIdx}}{{$line.RightIdx}}{{end}}"><span rel="{{if $line.RightIdx}}diff-{{Sha1 $file.Name}}R{{$line.RightIdx}}{{end}}"></span></td>
<td class="lines-type-marker lines-type-marker-new">{{if $line.RightIdx}}<span class="mono" data-type-marker="{{$line.GetLineTypeMarker}}"></span>{{end}}</td>
<td class="lines-code lines-code-new halfwidth">{{if and $.SignedUserID $line.CanComment $.PageIsPullFiles (not (eq .GetType 3))}}<a class="ui green button add-code-comment add-code-comment-right" data-path="{{$file.Name}}" data-side="right" data-idx="{{$line.RightIdx}}">+</a>{{end}}<span class="mono wrap{{if $highlightClass}} language-{{$highlightClass}}{{else}} nohighlight{{end}}">{{if $line.RightIdx}}{{$section.GetComputedInlineDiffFor $line}}{{end}}</span></td>
<td class="lines-code lines-code-new halfwidth">{{if and $.SignedUserID $line.CanComment $.PageIsPullFiles (not (eq .GetType 3))}}<a class="ui green button add-code-comment add-code-comment-right" data-path="{{$file.Name}}" data-side="right" data-idx="{{$line.RightIdx}}">+</a>{{end}}<span class="mono wrap">{{if $line.RightIdx}}{{$section.GetComputedInlineDiffFor $line}}{{end}}</span></td>
{{end}}
</tr>
{{if gt (len $line.Comments) 0}}

@ -1,5 +1,4 @@
{{$file := .file}}
{{$highlightClass := $file.GetHighlightClass}}
{{range $j, $section := $file.Sections}}
{{range $k, $line := $section.Lines}}
<tr class="{{DiffLineTypeToStr .GetType}}-code nl-{{$k}} ol-{{$k}}">
@ -20,7 +19,11 @@
<td class="lines-num lines-num-new" data-line-num="{{if $line.RightIdx}}{{$line.RightIdx}}{{end}}"><span rel="{{if $line.RightIdx}}diff-{{Sha1 $file.Name}}R{{$line.RightIdx}}{{end}}"></span></td>
{{end}}
<td class="lines-type-marker"><span class="mono" data-type-marker="{{$line.GetLineTypeMarker}}"></span></td>
<td class="lines-code{{if (not $line.RightIdx)}} lines-code-old{{end}}">{{if and $.root.SignedUserID $line.CanComment $.root.PageIsPullFiles}}<a class="ui green button add-code-comment add-code-comment-{{if $line.RightIdx}}right{{else}}left{{end}}" data-path="{{$file.Name}}" data-side="{{if $line.RightIdx}}right{{else}}left{{end}}" data-idx="{{if $line.RightIdx}}{{$line.RightIdx}}{{else}}{{$line.LeftIdx}}{{end}}">+</a>{{end}}<span class="mono wrap{{if $highlightClass}} language-{{$highlightClass}}{{else}} nohighlight{{end}}">{{$section.GetComputedInlineDiffFor $line}}</span></td>
{{if eq .GetType 4}}
<td class="chroma lines-code blob-hunk"><span class="mono wrap">{{$section.GetComputedInlineDiffFor $line}}</span></td>
{{else}}
<td class="chroma lines-code{{if (not $line.RightIdx)}} lines-code-old{{end}}">{{if and $.root.SignedUserID $line.CanComment $.root.PageIsPullFiles}}<a class="ui green button add-code-comment add-code-comment-{{if $line.RightIdx}}right{{else}}left{{end}}" data-path="{{$file.Name}}" data-side="{{if $line.RightIdx}}right{{else}}left{{end}}" data-idx="{{if $line.RightIdx}}{{$line.RightIdx}}{{else}}{{$line.LeftIdx}}{{end}}">+</a>{{end}}<span class="mono wrap">{{$section.GetComputedInlineDiffFor $line}}</span></td>
{{end}}
</tr>
{{if gt (len $line.Comments) 0}}
{{$resolved := (index $line.Comments 0).IsResolved}}

@ -42,7 +42,7 @@
<a href="{{EscapePound $.SourcePath}}/src/commit/{{$result.CommitID}}/{{EscapePound $result.Filename}}#L{{.}}"><span>{{.}}</span></a>
{{end}}
</td>
<td class="lines-code"><pre><code class="{{.HighlightClass}}"><ol class="linenums">{{.FormattedLines}}</ol></code></pre></td>
<td class="lines-code"><pre><code class="chroma"><ol class="linenums">{{.FormattedLines | Safe}}</ol></code></pre></td>
</tr>
</tbody>
</table>

@ -88,18 +88,30 @@
{{end}}
</div>
{{else if .FileSize}}
{{if .IsFileTooLarge}}
<table>
<tbody>
<tr>
{{if .IsFileTooLarge}}
<td><strong>{{.i18n.Tr "repo.file_too_large"}}</strong></td>
{{else}}
<td class="lines-num">{{.LineNums}}</td>
<td class="lines-code"><pre><code class="{{.HighlightClass}}"><ol class="linenums">{{.FileContent}}</ol></code></pre></td>
{{end}}
</tr>
</tbody>
</table>
{{else}}
<table>
<tbody>
{{range $line, $code := .FileContent}}
<tr>
<td id="L{{$line}}" class="lines-num">
<span id="L{{$line}}" data-line-number="{{$line}}"></span>
</td>
<td rel="L{{$line}}" class="lines-code chroma">
<pre><code>{{$code | Safe}}</code></pre>
</td>
</tr>
{{end}}
</tbody>
</table>
{{end}}
{{end}}
</div>
</div>

@ -0,0 +1,19 @@
# Binaries for programs and plugins
*.exe
*.dll
*.so
*.dylib
/cmd/chroma/chroma
# Test binary, build with `go test -c`
*.test
# Output of the go coverage tool, specifically when used with LiteIDE
*.out
# Project-local glide cache, RE: https://github.com/Masterminds/glide/issues/736
.glide/
_models/
_examples/

@ -0,0 +1,55 @@
run:
tests: true
skip-dirs:
- _examples
output:
print-issued-lines: false
linters:
enable-all: true
disable:
- maligned
- megacheck
- lll
- gocyclo
- dupl
- gochecknoglobals
- funlen
- godox
- wsl
- gomnd
- gocognit
linters-settings:
govet:
check-shadowing: true
gocyclo:
min-complexity: 10
dupl:
threshold: 100
goconst:
min-len: 8
min-occurrences: 3
issues:
max-per-linter: 0
max-same: 0
exclude-use-default: false
exclude:
# Captured by errcheck.
- '^(G104|G204):'
# Very commonly not checked.
- 'Error return value of .(.*\.Help|.*\.MarkFlagRequired|(os\.)?std(out|err)\..*|.*Close|.*Flush|os\.Remove(All)?|.*printf?|os\.(Un)?Setenv). is not checked'
- 'exported method (.*\.MarshalJSON|.*\.UnmarshalJSON|.*\.EntityURN|.*\.GoString|.*\.Pos) should have comment or be unexported'
- 'composite literal uses unkeyed fields'
- 'declaration of "err" shadows declaration'
- 'should not use dot imports'
- 'Potential file inclusion via variable'
- 'should have comment or be unexported'
- 'comment on exported var .* should be of the form'
- 'at least one file in a package should have a package comment'
- 'string literal contains the Unicode'
- 'methods on the same type should have the same receiver name'
- '_TokenType_name should be _TokenTypeName'
- '`_TokenType_map` should be `_TokenTypeMap`'

@ -0,0 +1,33 @@
project_name: chroma
release:
github:
owner: alecthomas
name: chroma
brews:
-
install: bin.install "chroma"
builds:
- goos:
- linux
- darwin
- windows
goarch:
- amd64
- "386"
goarm:
- "6"
main: ./cmd/chroma/main.go
ldflags: -s -w -X main.version={{.Version}} -X main.commit={{.Commit}} -X main.date={{.Date}}
binary: chroma
archives:
-