A fork of Gitea (see branch `mj`) adding Majority Judgment Polls š„· over Issues and Merge Requests. https://git.mieuxvoter.fr
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

1083 lines
30 KiB

Support unicode emojis and remove emojify.js (#11032) * Support unicode emojis and remove emojify.js This PR replaces all use of emojify.js and adds unicode emoji support to various areas of gitea. This works in a few ways: First it adds emoji parsing support into gitea itself. This allows us to * Render emojis from valid alias (:smile:) * Detect unicode emojis and let us put them in their own class with proper aria-labels and styling * Easily allow for custom "emoji" * Support all emoji rendering and features without javascript * Uses plain unicode and lets the system render in appropriate emoji font * Doesn't leave us relying on external sources for updates/fixes/features That same list of emoji is also used to create a json file which replaces the part of emojify.js that populates the emoji search tribute. This file is about 35KB with GZIP turned on and I've set it to load after the page renders to not hinder page load time (and this removes loading emojify.js also) For custom "emoji" it uses a pretty simple scheme of just looking for /emojis/img/name.png where name is something a user has put in the "allowed reactions" setting we already have. The gitea reaction that was previously hard coded into a forked copy of emojify.js is included and works as a custom reaction under this method. The emoji data sourced here is from https://github.com/github/gemoji which is the gem library Github uses for their emoji rendering (and a data source for other sites). So we should be able to easily render any emoji and :alias: that Github can, removing any errors from migrated content. They also update it as well, so we can sync when there are new unicode emoji lists released. I've included a slimmed down and slightly modified forked copy of https://github.com/knq/emoji to make up our own emoji module. The code is pretty straight forward and again allows us to have a lot of flexibility in what happens. I had seen a few comments about performance in some of the other threads if we render this ourselves, but there doesn't seem to be any issue here. In a test it can parse, convert, and render 1,000 emojis inside of a large markdown table in about 100ms on my laptop (which is many more emojis than will ever be in any normal issue). This also prevents any flickering and other weirdness from using javascript to render some things while using go for others. Not included here are image fall back URLS. I don't really think they are necessary for anything new being written in 2020. However, managing the emoji ourselves would allow us to add these as a feature later on if it seems necessary. Fixes: https://github.com/go-gitea/gitea/issues/9182 Fixes: https://github.com/go-gitea/gitea/issues/8974 Fixes: https://github.com/go-gitea/gitea/issues/8953 Fixes: https://github.com/go-gitea/gitea/issues/6628 Fixes: https://github.com/go-gitea/gitea/issues/5130 * add new shared function emojiHTML * don't increase emoji size in issue title * Update templates/repo/issue/view_content/add_reaction.tmpl Co-Authored-By: 6543 <6543@obermui.de> * Support for emoji rendering in various templates * Render code and review comments as they should be * Better way to handle mail subjects * insert unicode from tribute selection * Add template helper for plain text when needed * Use existing replace function I forgot about * Don't include emoji greater than Unicode Version 12 Only include emoji and aliases in JSON * Update build/generate-emoji.go * Tweak regex slightly to really match everything including random invisible characters. Run tests for every emoji we have * final updates * code review * code review * hard code gitea custom emoji to match previous behavior * Update .eslintrc Co-Authored-By: silverwind <me@silverwind.io> * disable preempt Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
3 years ago
Support unicode emojis and remove emojify.js (#11032) * Support unicode emojis and remove emojify.js This PR replaces all use of emojify.js and adds unicode emoji support to various areas of gitea. This works in a few ways: First it adds emoji parsing support into gitea itself. This allows us to * Render emojis from valid alias (:smile:) * Detect unicode emojis and let us put them in their own class with proper aria-labels and styling * Easily allow for custom "emoji" * Support all emoji rendering and features without javascript * Uses plain unicode and lets the system render in appropriate emoji font * Doesn't leave us relying on external sources for updates/fixes/features That same list of emoji is also used to create a json file which replaces the part of emojify.js that populates the emoji search tribute. This file is about 35KB with GZIP turned on and I've set it to load after the page renders to not hinder page load time (and this removes loading emojify.js also) For custom "emoji" it uses a pretty simple scheme of just looking for /emojis/img/name.png where name is something a user has put in the "allowed reactions" setting we already have. The gitea reaction that was previously hard coded into a forked copy of emojify.js is included and works as a custom reaction under this method. The emoji data sourced here is from https://github.com/github/gemoji which is the gem library Github uses for their emoji rendering (and a data source for other sites). So we should be able to easily render any emoji and :alias: that Github can, removing any errors from migrated content. They also update it as well, so we can sync when there are new unicode emoji lists released. I've included a slimmed down and slightly modified forked copy of https://github.com/knq/emoji to make up our own emoji module. The code is pretty straight forward and again allows us to have a lot of flexibility in what happens. I had seen a few comments about performance in some of the other threads if we render this ourselves, but there doesn't seem to be any issue here. In a test it can parse, convert, and render 1,000 emojis inside of a large markdown table in about 100ms on my laptop (which is many more emojis than will ever be in any normal issue). This also prevents any flickering and other weirdness from using javascript to render some things while using go for others. Not included here are image fall back URLS. I don't really think they are necessary for anything new being written in 2020. However, managing the emoji ourselves would allow us to add these as a feature later on if it seems necessary. Fixes: https://github.com/go-gitea/gitea/issues/9182 Fixes: https://github.com/go-gitea/gitea/issues/8974 Fixes: https://github.com/go-gitea/gitea/issues/8953 Fixes: https://github.com/go-gitea/gitea/issues/6628 Fixes: https://github.com/go-gitea/gitea/issues/5130 * add new shared function emojiHTML * don't increase emoji size in issue title * Update templates/repo/issue/view_content/add_reaction.tmpl Co-Authored-By: 6543 <6543@obermui.de> * Support for emoji rendering in various templates * Render code and review comments as they should be * Better way to handle mail subjects * insert unicode from tribute selection * Add template helper for plain text when needed * Use existing replace function I forgot about * Don't include emoji greater than Unicode Version 12 Only include emoji and aliases in JSON * Update build/generate-emoji.go * Tweak regex slightly to really match everything including random invisible characters. Run tests for every emoji we have * final updates * code review * code review * hard code gitea custom emoji to match previous behavior * Update .eslintrc Co-Authored-By: silverwind <me@silverwind.io> * disable preempt Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
3 years ago
Check commit message hashes before making links (#7713) * Check commit message hashes before making links Previously, when formatting commit messages, anything that looked like SHA1 hashes was turned into a link using regex. This meant that certain phrases or numbers such as `777777` or `deadbeef` could be recognized as a commit even if the repository has no commit with those hashes. This change will make it so that anything that looks like a SHA1 hash using regex will then also be checked to ensure that there is a commit in the repository with that hash before making a link. Signed-off-by: Gary Kim <gary@garykim.dev> * Use gogit to check if commit exists This commit modifies the commit hash check in the render for commit messages to use gogit for better performance. Signed-off-by: Gary Kim <gary@garykim.dev> * Make code cleaner Signed-off-by: Gary Kim <gary@garykim.dev> * Use rev-parse to check if commit exists Signed-off-by: Gary Kim <gary@garykim.dev> * Add and modify tests for checking hashes in html link rendering Signed-off-by: Gary Kim <gary@garykim.dev> * Return error in sha1CurrentPatternProcessor Co-Authored-By: mrsdizzie <info@mrsdizzie.com> * Import Gitea log module Signed-off-by: Gary Kim <gary@garykim.dev> * Revert "Return error in sha1CurrentPatternProcessor" This reverts commit 28f561cac46ef7e51aa26aefcbe9aca4671366a6. Signed-off-by: Gary Kim <gary@garykim.dev> * Add debug logging to sha1CurrentPatternProcessor This will log errors by the git command run in sha1CurrentPatternProcessor if the error is one that was unexpected. Signed-off-by: Gary Kim <gary@garykim.dev>
3 years ago
Support unicode emojis and remove emojify.js (#11032) * Support unicode emojis and remove emojify.js This PR replaces all use of emojify.js and adds unicode emoji support to various areas of gitea. This works in a few ways: First it adds emoji parsing support into gitea itself. This allows us to * Render emojis from valid alias (:smile:) * Detect unicode emojis and let us put them in their own class with proper aria-labels and styling * Easily allow for custom "emoji" * Support all emoji rendering and features without javascript * Uses plain unicode and lets the system render in appropriate emoji font * Doesn't leave us relying on external sources for updates/fixes/features That same list of emoji is also used to create a json file which replaces the part of emojify.js that populates the emoji search tribute. This file is about 35KB with GZIP turned on and I've set it to load after the page renders to not hinder page load time (and this removes loading emojify.js also) For custom "emoji" it uses a pretty simple scheme of just looking for /emojis/img/name.png where name is something a user has put in the "allowed reactions" setting we already have. The gitea reaction that was previously hard coded into a forked copy of emojify.js is included and works as a custom reaction under this method. The emoji data sourced here is from https://github.com/github/gemoji which is the gem library Github uses for their emoji rendering (and a data source for other sites). So we should be able to easily render any emoji and :alias: that Github can, removing any errors from migrated content. They also update it as well, so we can sync when there are new unicode emoji lists released. I've included a slimmed down and slightly modified forked copy of https://github.com/knq/emoji to make up our own emoji module. The code is pretty straight forward and again allows us to have a lot of flexibility in what happens. I had seen a few comments about performance in some of the other threads if we render this ourselves, but there doesn't seem to be any issue here. In a test it can parse, convert, and render 1,000 emojis inside of a large markdown table in about 100ms on my laptop (which is many more emojis than will ever be in any normal issue). This also prevents any flickering and other weirdness from using javascript to render some things while using go for others. Not included here are image fall back URLS. I don't really think they are necessary for anything new being written in 2020. However, managing the emoji ourselves would allow us to add these as a feature later on if it seems necessary. Fixes: https://github.com/go-gitea/gitea/issues/9182 Fixes: https://github.com/go-gitea/gitea/issues/8974 Fixes: https://github.com/go-gitea/gitea/issues/8953 Fixes: https://github.com/go-gitea/gitea/issues/6628 Fixes: https://github.com/go-gitea/gitea/issues/5130 * add new shared function emojiHTML * don't increase emoji size in issue title * Update templates/repo/issue/view_content/add_reaction.tmpl Co-Authored-By: 6543 <6543@obermui.de> * Support for emoji rendering in various templates * Render code and review comments as they should be * Better way to handle mail subjects * insert unicode from tribute selection * Add template helper for plain text when needed * Use existing replace function I forgot about * Don't include emoji greater than Unicode Version 12 Only include emoji and aliases in JSON * Update build/generate-emoji.go * Tweak regex slightly to really match everything including random invisible characters. Run tests for every emoji we have * final updates * code review * code review * hard code gitea custom emoji to match previous behavior * Update .eslintrc Co-Authored-By: silverwind <me@silverwind.io> * disable preempt Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
3 years ago
Support unicode emojis and remove emojify.js (#11032) * Support unicode emojis and remove emojify.js This PR replaces all use of emojify.js and adds unicode emoji support to various areas of gitea. This works in a few ways: First it adds emoji parsing support into gitea itself. This allows us to * Render emojis from valid alias (:smile:) * Detect unicode emojis and let us put them in their own class with proper aria-labels and styling * Easily allow for custom "emoji" * Support all emoji rendering and features without javascript * Uses plain unicode and lets the system render in appropriate emoji font * Doesn't leave us relying on external sources for updates/fixes/features That same list of emoji is also used to create a json file which replaces the part of emojify.js that populates the emoji search tribute. This file is about 35KB with GZIP turned on and I've set it to load after the page renders to not hinder page load time (and this removes loading emojify.js also) For custom "emoji" it uses a pretty simple scheme of just looking for /emojis/img/name.png where name is something a user has put in the "allowed reactions" setting we already have. The gitea reaction that was previously hard coded into a forked copy of emojify.js is included and works as a custom reaction under this method. The emoji data sourced here is from https://github.com/github/gemoji which is the gem library Github uses for their emoji rendering (and a data source for other sites). So we should be able to easily render any emoji and :alias: that Github can, removing any errors from migrated content. They also update it as well, so we can sync when there are new unicode emoji lists released. I've included a slimmed down and slightly modified forked copy of https://github.com/knq/emoji to make up our own emoji module. The code is pretty straight forward and again allows us to have a lot of flexibility in what happens. I had seen a few comments about performance in some of the other threads if we render this ourselves, but there doesn't seem to be any issue here. In a test it can parse, convert, and render 1,000 emojis inside of a large markdown table in about 100ms on my laptop (which is many more emojis than will ever be in any normal issue). This also prevents any flickering and other weirdness from using javascript to render some things while using go for others. Not included here are image fall back URLS. I don't really think they are necessary for anything new being written in 2020. However, managing the emoji ourselves would allow us to add these as a feature later on if it seems necessary. Fixes: https://github.com/go-gitea/gitea/issues/9182 Fixes: https://github.com/go-gitea/gitea/issues/8974 Fixes: https://github.com/go-gitea/gitea/issues/8953 Fixes: https://github.com/go-gitea/gitea/issues/6628 Fixes: https://github.com/go-gitea/gitea/issues/5130 * add new shared function emojiHTML * don't increase emoji size in issue title * Update templates/repo/issue/view_content/add_reaction.tmpl Co-Authored-By: 6543 <6543@obermui.de> * Support for emoji rendering in various templates * Render code and review comments as they should be * Better way to handle mail subjects * insert unicode from tribute selection * Add template helper for plain text when needed * Use existing replace function I forgot about * Don't include emoji greater than Unicode Version 12 Only include emoji and aliases in JSON * Update build/generate-emoji.go * Tweak regex slightly to really match everything including random invisible characters. Run tests for every emoji we have * final updates * code review * code review * hard code gitea custom emoji to match previous behavior * Update .eslintrc Co-Authored-By: silverwind <me@silverwind.io> * disable preempt Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
3 years ago
Support unicode emojis and remove emojify.js (#11032) * Support unicode emojis and remove emojify.js This PR replaces all use of emojify.js and adds unicode emoji support to various areas of gitea. This works in a few ways: First it adds emoji parsing support into gitea itself. This allows us to * Render emojis from valid alias (:smile:) * Detect unicode emojis and let us put them in their own class with proper aria-labels and styling * Easily allow for custom "emoji" * Support all emoji rendering and features without javascript * Uses plain unicode and lets the system render in appropriate emoji font * Doesn't leave us relying on external sources for updates/fixes/features That same list of emoji is also used to create a json file which replaces the part of emojify.js that populates the emoji search tribute. This file is about 35KB with GZIP turned on and I've set it to load after the page renders to not hinder page load time (and this removes loading emojify.js also) For custom "emoji" it uses a pretty simple scheme of just looking for /emojis/img/name.png where name is something a user has put in the "allowed reactions" setting we already have. The gitea reaction that was previously hard coded into a forked copy of emojify.js is included and works as a custom reaction under this method. The emoji data sourced here is from https://github.com/github/gemoji which is the gem library Github uses for their emoji rendering (and a data source for other sites). So we should be able to easily render any emoji and :alias: that Github can, removing any errors from migrated content. They also update it as well, so we can sync when there are new unicode emoji lists released. I've included a slimmed down and slightly modified forked copy of https://github.com/knq/emoji to make up our own emoji module. The code is pretty straight forward and again allows us to have a lot of flexibility in what happens. I had seen a few comments about performance in some of the other threads if we render this ourselves, but there doesn't seem to be any issue here. In a test it can parse, convert, and render 1,000 emojis inside of a large markdown table in about 100ms on my laptop (which is many more emojis than will ever be in any normal issue). This also prevents any flickering and other weirdness from using javascript to render some things while using go for others. Not included here are image fall back URLS. I don't really think they are necessary for anything new being written in 2020. However, managing the emoji ourselves would allow us to add these as a feature later on if it seems necessary. Fixes: https://github.com/go-gitea/gitea/issues/9182 Fixes: https://github.com/go-gitea/gitea/issues/8974 Fixes: https://github.com/go-gitea/gitea/issues/8953 Fixes: https://github.com/go-gitea/gitea/issues/6628 Fixes: https://github.com/go-gitea/gitea/issues/5130 * add new shared function emojiHTML * don't increase emoji size in issue title * Update templates/repo/issue/view_content/add_reaction.tmpl Co-Authored-By: 6543 <6543@obermui.de> * Support for emoji rendering in various templates * Render code and review comments as they should be * Better way to handle mail subjects * insert unicode from tribute selection * Add template helper for plain text when needed * Use existing replace function I forgot about * Don't include emoji greater than Unicode Version 12 Only include emoji and aliases in JSON * Update build/generate-emoji.go * Tweak regex slightly to really match everything including random invisible characters. Run tests for every emoji we have * final updates * code review * code review * hard code gitea custom emoji to match previous behavior * Update .eslintrc Co-Authored-By: silverwind <me@silverwind.io> * disable preempt Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
3 years ago
Support unicode emojis and remove emojify.js (#11032) * Support unicode emojis and remove emojify.js This PR replaces all use of emojify.js and adds unicode emoji support to various areas of gitea. This works in a few ways: First it adds emoji parsing support into gitea itself. This allows us to * Render emojis from valid alias (:smile:) * Detect unicode emojis and let us put them in their own class with proper aria-labels and styling * Easily allow for custom "emoji" * Support all emoji rendering and features without javascript * Uses plain unicode and lets the system render in appropriate emoji font * Doesn't leave us relying on external sources for updates/fixes/features That same list of emoji is also used to create a json file which replaces the part of emojify.js that populates the emoji search tribute. This file is about 35KB with GZIP turned on and I've set it to load after the page renders to not hinder page load time (and this removes loading emojify.js also) For custom "emoji" it uses a pretty simple scheme of just looking for /emojis/img/name.png where name is something a user has put in the "allowed reactions" setting we already have. The gitea reaction that was previously hard coded into a forked copy of emojify.js is included and works as a custom reaction under this method. The emoji data sourced here is from https://github.com/github/gemoji which is the gem library Github uses for their emoji rendering (and a data source for other sites). So we should be able to easily render any emoji and :alias: that Github can, removing any errors from migrated content. They also update it as well, so we can sync when there are new unicode emoji lists released. I've included a slimmed down and slightly modified forked copy of https://github.com/knq/emoji to make up our own emoji module. The code is pretty straight forward and again allows us to have a lot of flexibility in what happens. I had seen a few comments about performance in some of the other threads if we render this ourselves, but there doesn't seem to be any issue here. In a test it can parse, convert, and render 1,000 emojis inside of a large markdown table in about 100ms on my laptop (which is many more emojis than will ever be in any normal issue). This also prevents any flickering and other weirdness from using javascript to render some things while using go for others. Not included here are image fall back URLS. I don't really think they are necessary for anything new being written in 2020. However, managing the emoji ourselves would allow us to add these as a feature later on if it seems necessary. Fixes: https://github.com/go-gitea/gitea/issues/9182 Fixes: https://github.com/go-gitea/gitea/issues/8974 Fixes: https://github.com/go-gitea/gitea/issues/8953 Fixes: https://github.com/go-gitea/gitea/issues/6628 Fixes: https://github.com/go-gitea/gitea/issues/5130 * add new shared function emojiHTML * don't increase emoji size in issue title * Update templates/repo/issue/view_content/add_reaction.tmpl Co-Authored-By: 6543 <6543@obermui.de> * Support for emoji rendering in various templates * Render code and review comments as they should be * Better way to handle mail subjects * insert unicode from tribute selection * Add template helper for plain text when needed * Use existing replace function I forgot about * Don't include emoji greater than Unicode Version 12 Only include emoji and aliases in JSON * Update build/generate-emoji.go * Tweak regex slightly to really match everything including random invisible characters. Run tests for every emoji we have * final updates * code review * code review * hard code gitea custom emoji to match previous behavior * Update .eslintrc Co-Authored-By: silverwind <me@silverwind.io> * disable preempt Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
3 years ago
Support unicode emojis and remove emojify.js (#11032) * Support unicode emojis and remove emojify.js This PR replaces all use of emojify.js and adds unicode emoji support to various areas of gitea. This works in a few ways: First it adds emoji parsing support into gitea itself. This allows us to * Render emojis from valid alias (:smile:) * Detect unicode emojis and let us put them in their own class with proper aria-labels and styling * Easily allow for custom "emoji" * Support all emoji rendering and features without javascript * Uses plain unicode and lets the system render in appropriate emoji font * Doesn't leave us relying on external sources for updates/fixes/features That same list of emoji is also used to create a json file which replaces the part of emojify.js that populates the emoji search tribute. This file is about 35KB with GZIP turned on and I've set it to load after the page renders to not hinder page load time (and this removes loading emojify.js also) For custom "emoji" it uses a pretty simple scheme of just looking for /emojis/img/name.png where name is something a user has put in the "allowed reactions" setting we already have. The gitea reaction that was previously hard coded into a forked copy of emojify.js is included and works as a custom reaction under this method. The emoji data sourced here is from https://github.com/github/gemoji which is the gem library Github uses for their emoji rendering (and a data source for other sites). So we should be able to easily render any emoji and :alias: that Github can, removing any errors from migrated content. They also update it as well, so we can sync when there are new unicode emoji lists released. I've included a slimmed down and slightly modified forked copy of https://github.com/knq/emoji to make up our own emoji module. The code is pretty straight forward and again allows us to have a lot of flexibility in what happens. I had seen a few comments about performance in some of the other threads if we render this ourselves, but there doesn't seem to be any issue here. In a test it can parse, convert, and render 1,000 emojis inside of a large markdown table in about 100ms on my laptop (which is many more emojis than will ever be in any normal issue). This also prevents any flickering and other weirdness from using javascript to render some things while using go for others. Not included here are image fall back URLS. I don't really think they are necessary for anything new being written in 2020. However, managing the emoji ourselves would allow us to add these as a feature later on if it seems necessary. Fixes: https://github.com/go-gitea/gitea/issues/9182 Fixes: https://github.com/go-gitea/gitea/issues/8974 Fixes: https://github.com/go-gitea/gitea/issues/8953 Fixes: https://github.com/go-gitea/gitea/issues/6628 Fixes: https://github.com/go-gitea/gitea/issues/5130 * add new shared function emojiHTML * don't increase emoji size in issue title * Update templates/repo/issue/view_content/add_reaction.tmpl Co-Authored-By: 6543 <6543@obermui.de> * Support for emoji rendering in various templates * Render code and review comments as they should be * Better way to handle mail subjects * insert unicode from tribute selection * Add template helper for plain text when needed * Use existing replace function I forgot about * Don't include emoji greater than Unicode Version 12 Only include emoji and aliases in JSON * Update build/generate-emoji.go * Tweak regex slightly to really match everything including random invisible characters. Run tests for every emoji we have * final updates * code review * code review * hard code gitea custom emoji to match previous behavior * Update .eslintrc Co-Authored-By: silverwind <me@silverwind.io> * disable preempt Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
3 years ago
Support unicode emojis and remove emojify.js (#11032) * Support unicode emojis and remove emojify.js This PR replaces all use of emojify.js and adds unicode emoji support to various areas of gitea. This works in a few ways: First it adds emoji parsing support into gitea itself. This allows us to * Render emojis from valid alias (:smile:) * Detect unicode emojis and let us put them in their own class with proper aria-labels and styling * Easily allow for custom "emoji" * Support all emoji rendering and features without javascript * Uses plain unicode and lets the system render in appropriate emoji font * Doesn't leave us relying on external sources for updates/fixes/features That same list of emoji is also used to create a json file which replaces the part of emojify.js that populates the emoji search tribute. This file is about 35KB with GZIP turned on and I've set it to load after the page renders to not hinder page load time (and this removes loading emojify.js also) For custom "emoji" it uses a pretty simple scheme of just looking for /emojis/img/name.png where name is something a user has put in the "allowed reactions" setting we already have. The gitea reaction that was previously hard coded into a forked copy of emojify.js is included and works as a custom reaction under this method. The emoji data sourced here is from https://github.com/github/gemoji which is the gem library Github uses for their emoji rendering (and a data source for other sites). So we should be able to easily render any emoji and :alias: that Github can, removing any errors from migrated content. They also update it as well, so we can sync when there are new unicode emoji lists released. I've included a slimmed down and slightly modified forked copy of https://github.com/knq/emoji to make up our own emoji module. The code is pretty straight forward and again allows us to have a lot of flexibility in what happens. I had seen a few comments about performance in some of the other threads if we render this ourselves, but there doesn't seem to be any issue here. In a test it can parse, convert, and render 1,000 emojis inside of a large markdown table in about 100ms on my laptop (which is many more emojis than will ever be in any normal issue). This also prevents any flickering and other weirdness from using javascript to render some things while using go for others. Not included here are image fall back URLS. I don't really think they are necessary for anything new being written in 2020. However, managing the emoji ourselves would allow us to add these as a feature later on if it seems necessary. Fixes: https://github.com/go-gitea/gitea/issues/9182 Fixes: https://github.com/go-gitea/gitea/issues/8974 Fixes: https://github.com/go-gitea/gitea/issues/8953 Fixes: https://github.com/go-gitea/gitea/issues/6628 Fixes: https://github.com/go-gitea/gitea/issues/5130 * add new shared function emojiHTML * don't increase emoji size in issue title * Update templates/repo/issue/view_content/add_reaction.tmpl Co-Authored-By: 6543 <6543@obermui.de> * Support for emoji rendering in various templates * Render code and review comments as they should be * Better way to handle mail subjects * insert unicode from tribute selection * Add template helper for plain text when needed * Use existing replace function I forgot about * Don't include emoji greater than Unicode Version 12 Only include emoji and aliases in JSON * Update build/generate-emoji.go * Tweak regex slightly to really match everything including random invisible characters. Run tests for every emoji we have * final updates * code review * code review * hard code gitea custom emoji to match previous behavior * Update .eslintrc Co-Authored-By: silverwind <me@silverwind.io> * disable preempt Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
3 years ago
Support unicode emojis and remove emojify.js (#11032) * Support unicode emojis and remove emojify.js This PR replaces all use of emojify.js and adds unicode emoji support to various areas of gitea. This works in a few ways: First it adds emoji parsing support into gitea itself. This allows us to * Render emojis from valid alias (:smile:) * Detect unicode emojis and let us put them in their own class with proper aria-labels and styling * Easily allow for custom "emoji" * Support all emoji rendering and features without javascript * Uses plain unicode and lets the system render in appropriate emoji font * Doesn't leave us relying on external sources for updates/fixes/features That same list of emoji is also used to create a json file which replaces the part of emojify.js that populates the emoji search tribute. This file is about 35KB with GZIP turned on and I've set it to load after the page renders to not hinder page load time (and this removes loading emojify.js also) For custom "emoji" it uses a pretty simple scheme of just looking for /emojis/img/name.png where name is something a user has put in the "allowed reactions" setting we already have. The gitea reaction that was previously hard coded into a forked copy of emojify.js is included and works as a custom reaction under this method. The emoji data sourced here is from https://github.com/github/gemoji which is the gem library Github uses for their emoji rendering (and a data source for other sites). So we should be able to easily render any emoji and :alias: that Github can, removing any errors from migrated content. They also update it as well, so we can sync when there are new unicode emoji lists released. I've included a slimmed down and slightly modified forked copy of https://github.com/knq/emoji to make up our own emoji module. The code is pretty straight forward and again allows us to have a lot of flexibility in what happens. I had seen a few comments about performance in some of the other threads if we render this ourselves, but there doesn't seem to be any issue here. In a test it can parse, convert, and render 1,000 emojis inside of a large markdown table in about 100ms on my laptop (which is many more emojis than will ever be in any normal issue). This also prevents any flickering and other weirdness from using javascript to render some things while using go for others. Not included here are image fall back URLS. I don't really think they are necessary for anything new being written in 2020. However, managing the emoji ourselves would allow us to add these as a feature later on if it seems necessary. Fixes: https://github.com/go-gitea/gitea/issues/9182 Fixes: https://github.com/go-gitea/gitea/issues/8974 Fixes: https://github.com/go-gitea/gitea/issues/8953 Fixes: https://github.com/go-gitea/gitea/issues/6628 Fixes: https://github.com/go-gitea/gitea/issues/5130 * add new shared function emojiHTML * don't increase emoji size in issue title * Update templates/repo/issue/view_content/add_reaction.tmpl Co-Authored-By: 6543 <6543@obermui.de> * Support for emoji rendering in various templates * Render code and review comments as they should be * Better way to handle mail subjects * insert unicode from tribute selection * Add template helper for plain text when needed * Use existing replace function I forgot about * Don't include emoji greater than Unicode Version 12 Only include emoji and aliases in JSON * Update build/generate-emoji.go * Tweak regex slightly to really match everything including random invisible characters. Run tests for every emoji we have * final updates * code review * code review * hard code gitea custom emoji to match previous behavior * Update .eslintrc Co-Authored-By: silverwind <me@silverwind.io> * disable preempt Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
3 years ago
Support unicode emojis and remove emojify.js (#11032) * Support unicode emojis and remove emojify.js This PR replaces all use of emojify.js and adds unicode emoji support to various areas of gitea. This works in a few ways: First it adds emoji parsing support into gitea itself. This allows us to * Render emojis from valid alias (:smile:) * Detect unicode emojis and let us put them in their own class with proper aria-labels and styling * Easily allow for custom "emoji" * Support all emoji rendering and features without javascript * Uses plain unicode and lets the system render in appropriate emoji font * Doesn't leave us relying on external sources for updates/fixes/features That same list of emoji is also used to create a json file which replaces the part of emojify.js that populates the emoji search tribute. This file is about 35KB with GZIP turned on and I've set it to load after the page renders to not hinder page load time (and this removes loading emojify.js also) For custom "emoji" it uses a pretty simple scheme of just looking for /emojis/img/name.png where name is something a user has put in the "allowed reactions" setting we already have. The gitea reaction that was previously hard coded into a forked copy of emojify.js is included and works as a custom reaction under this method. The emoji data sourced here is from https://github.com/github/gemoji which is the gem library Github uses for their emoji rendering (and a data source for other sites). So we should be able to easily render any emoji and :alias: that Github can, removing any errors from migrated content. They also update it as well, so we can sync when there are new unicode emoji lists released. I've included a slimmed down and slightly modified forked copy of https://github.com/knq/emoji to make up our own emoji module. The code is pretty straight forward and again allows us to have a lot of flexibility in what happens. I had seen a few comments about performance in some of the other threads if we render this ourselves, but there doesn't seem to be any issue here. In a test it can parse, convert, and render 1,000 emojis inside of a large markdown table in about 100ms on my laptop (which is many more emojis than will ever be in any normal issue). This also prevents any flickering and other weirdness from using javascript to render some things while using go for others. Not included here are image fall back URLS. I don't really think they are necessary for anything new being written in 2020. However, managing the emoji ourselves would allow us to add these as a feature later on if it seems necessary. Fixes: https://github.com/go-gitea/gitea/issues/9182 Fixes: https://github.com/go-gitea/gitea/issues/8974 Fixes: https://github.com/go-gitea/gitea/issues/8953 Fixes: https://github.com/go-gitea/gitea/issues/6628 Fixes: https://github.com/go-gitea/gitea/issues/5130 * add new shared function emojiHTML * don't increase emoji size in issue title * Update templates/repo/issue/view_content/add_reaction.tmpl Co-Authored-By: 6543 <6543@obermui.de> * Support for emoji rendering in various templates * Render code and review comments as they should be * Better way to handle mail subjects * insert unicode from tribute selection * Add template helper for plain text when needed * Use existing replace function I forgot about * Don't include emoji greater than Unicode Version 12 Only include emoji and aliases in JSON * Update build/generate-emoji.go * Tweak regex slightly to really match everything including random invisible characters. Run tests for every emoji we have * final updates * code review * code review * hard code gitea custom emoji to match previous behavior * Update .eslintrc Co-Authored-By: silverwind <me@silverwind.io> * disable preempt Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
3 years ago
Support unicode emojis and remove emojify.js (#11032) * Support unicode emojis and remove emojify.js This PR replaces all use of emojify.js and adds unicode emoji support to various areas of gitea. This works in a few ways: First it adds emoji parsing support into gitea itself. This allows us to * Render emojis from valid alias (:smile:) * Detect unicode emojis and let us put them in their own class with proper aria-labels and styling * Easily allow for custom "emoji" * Support all emoji rendering and features without javascript * Uses plain unicode and lets the system render in appropriate emoji font * Doesn't leave us relying on external sources for updates/fixes/features That same list of emoji is also used to create a json file which replaces the part of emojify.js that populates the emoji search tribute. This file is about 35KB with GZIP turned on and I've set it to load after the page renders to not hinder page load time (and this removes loading emojify.js also) For custom "emoji" it uses a pretty simple scheme of just looking for /emojis/img/name.png where name is something a user has put in the "allowed reactions" setting we already have. The gitea reaction that was previously hard coded into a forked copy of emojify.js is included and works as a custom reaction under this method. The emoji data sourced here is from https://github.com/github/gemoji which is the gem library Github uses for their emoji rendering (and a data source for other sites). So we should be able to easily render any emoji and :alias: that Github can, removing any errors from migrated content. They also update it as well, so we can sync when there are new unicode emoji lists released. I've included a slimmed down and slightly modified forked copy of https://github.com/knq/emoji to make up our own emoji module. The code is pretty straight forward and again allows us to have a lot of flexibility in what happens. I had seen a few comments about performance in some of the other threads if we render this ourselves, but there doesn't seem to be any issue here. In a test it can parse, convert, and render 1,000 emojis inside of a large markdown table in about 100ms on my laptop (which is many more emojis than will ever be in any normal issue). This also prevents any flickering and other weirdness from using javascript to render some things while using go for others. Not included here are image fall back URLS. I don't really think they are necessary for anything new being written in 2020. However, managing the emoji ourselves would allow us to add these as a feature later on if it seems necessary. Fixes: https://github.com/go-gitea/gitea/issues/9182 Fixes: https://github.com/go-gitea/gitea/issues/8974 Fixes: https://github.com/go-gitea/gitea/issues/8953 Fixes: https://github.com/go-gitea/gitea/issues/6628 Fixes: https://github.com/go-gitea/gitea/issues/5130 * add new shared function emojiHTML * don't increase emoji size in issue title * Update templates/repo/issue/view_content/add_reaction.tmpl Co-Authored-By: 6543 <6543@obermui.de> * Support for emoji rendering in various templates * Render code and review comments as they should be * Better way to handle mail subjects * insert unicode from tribute selection * Add template helper for plain text when needed * Use existing replace function I forgot about * Don't include emoji greater than Unicode Version 12 Only include emoji and aliases in JSON * Update build/generate-emoji.go * Tweak regex slightly to really match everything including random invisible characters. Run tests for every emoji we have * final updates * code review * code review * hard code gitea custom emoji to match previous behavior * Update .eslintrc Co-Authored-By: silverwind <me@silverwind.io> * disable preempt Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
3 years ago
Support unicode emojis and remove emojify.js (#11032) * Support unicode emojis and remove emojify.js This PR replaces all use of emojify.js and adds unicode emoji support to various areas of gitea. This works in a few ways: First it adds emoji parsing support into gitea itself. This allows us to * Render emojis from valid alias (:smile:) * Detect unicode emojis and let us put them in their own class with proper aria-labels and styling * Easily allow for custom "emoji" * Support all emoji rendering and features without javascript * Uses plain unicode and lets the system render in appropriate emoji font * Doesn't leave us relying on external sources for updates/fixes/features That same list of emoji is also used to create a json file which replaces the part of emojify.js that populates the emoji search tribute. This file is about 35KB with GZIP turned on and I've set it to load after the page renders to not hinder page load time (and this removes loading emojify.js also) For custom "emoji" it uses a pretty simple scheme of just looking for /emojis/img/name.png where name is something a user has put in the "allowed reactions" setting we already have. The gitea reaction that was previously hard coded into a forked copy of emojify.js is included and works as a custom reaction under this method. The emoji data sourced here is from https://github.com/github/gemoji which is the gem library Github uses for their emoji rendering (and a data source for other sites). So we should be able to easily render any emoji and :alias: that Github can, removing any errors from migrated content. They also update it as well, so we can sync when there are new unicode emoji lists released. I've included a slimmed down and slightly modified forked copy of https://github.com/knq/emoji to make up our own emoji module. The code is pretty straight forward and again allows us to have a lot of flexibility in what happens. I had seen a few comments about performance in some of the other threads if we render this ourselves, but there doesn't seem to be any issue here. In a test it can parse, convert, and render 1,000 emojis inside of a large markdown table in about 100ms on my laptop (which is many more emojis than will ever be in any normal issue). This also prevents any flickering and other weirdness from using javascript to render some things while using go for others. Not included here are image fall back URLS. I don't really think they are necessary for anything new being written in 2020. However, managing the emoji ourselves would allow us to add these as a feature later on if it seems necessary. Fixes: https://github.com/go-gitea/gitea/issues/9182 Fixes: https://github.com/go-gitea/gitea/issues/8974 Fixes: https://github.com/go-gitea/gitea/issues/8953 Fixes: https://github.com/go-gitea/gitea/issues/6628 Fixes: https://github.com/go-gitea/gitea/issues/5130 * add new shared function emojiHTML * don't increase emoji size in issue title * Update templates/repo/issue/view_content/add_reaction.tmpl Co-Authored-By: 6543 <6543@obermui.de> * Support for emoji rendering in various templates * Render code and review comments as they should be * Better way to handle mail subjects * insert unicode from tribute selection * Add template helper for plain text when needed * Use existing replace function I forgot about * Don't include emoji greater than Unicode Version 12 Only include emoji and aliases in JSON * Update build/generate-emoji.go * Tweak regex slightly to really match everything including random invisible characters. Run tests for every emoji we have * final updates * code review * code review * hard code gitea custom emoji to match previous behavior * Update .eslintrc Co-Authored-By: silverwind <me@silverwind.io> * disable preempt Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
3 years ago
Support unicode emojis and remove emojify.js (#11032) * Support unicode emojis and remove emojify.js This PR replaces all use of emojify.js and adds unicode emoji support to various areas of gitea. This works in a few ways: First it adds emoji parsing support into gitea itself. This allows us to * Render emojis from valid alias (:smile:) * Detect unicode emojis and let us put them in their own class with proper aria-labels and styling * Easily allow for custom "emoji" * Support all emoji rendering and features without javascript * Uses plain unicode and lets the system render in appropriate emoji font * Doesn't leave us relying on external sources for updates/fixes/features That same list of emoji is also used to create a json file which replaces the part of emojify.js that populates the emoji search tribute. This file is about 35KB with GZIP turned on and I've set it to load after the page renders to not hinder page load time (and this removes loading emojify.js also) For custom "emoji" it uses a pretty simple scheme of just looking for /emojis/img/name.png where name is something a user has put in the "allowed reactions" setting we already have. The gitea reaction that was previously hard coded into a forked copy of emojify.js is included and works as a custom reaction under this method. The emoji data sourced here is from https://github.com/github/gemoji which is the gem library Github uses for their emoji rendering (and a data source for other sites). So we should be able to easily render any emoji and :alias: that Github can, removing any errors from migrated content. They also update it as well, so we can sync when there are new unicode emoji lists released. I've included a slimmed down and slightly modified forked copy of https://github.com/knq/emoji to make up our own emoji module. The code is pretty straight forward and again allows us to have a lot of flexibility in what happens. I had seen a few comments about performance in some of the other threads if we render this ourselves, but there doesn't seem to be any issue here. In a test it can parse, convert, and render 1,000 emojis inside of a large markdown table in about 100ms on my laptop (which is many more emojis than will ever be in any normal issue). This also prevents any flickering and other weirdness from using javascript to render some things while using go for others. Not included here are image fall back URLS. I don't really think they are necessary for anything new being written in 2020. However, managing the emoji ourselves would allow us to add these as a feature later on if it seems necessary. Fixes: https://github.com/go-gitea/gitea/issues/9182 Fixes: https://github.com/go-gitea/gitea/issues/8974 Fixes: https://github.com/go-gitea/gitea/issues/8953 Fixes: https://github.com/go-gitea/gitea/issues/6628 Fixes: https://github.com/go-gitea/gitea/issues/5130 * add new shared function emojiHTML * don't increase emoji size in issue title * Update templates/repo/issue/view_content/add_reaction.tmpl Co-Authored-By: 6543 <6543@obermui.de> * Support for emoji rendering in various templates * Render code and review comments as they should be * Better way to handle mail subjects * insert unicode from tribute selection * Add template helper for plain text when needed * Use existing replace function I forgot about * Don't include emoji greater than Unicode Version 12 Only include emoji and aliases in JSON * Update build/generate-emoji.go * Tweak regex slightly to really match everything including random invisible characters. Run tests for every emoji we have * final updates * code review * code review * hard code gitea custom emoji to match previous behavior * Update .eslintrc Co-Authored-By: silverwind <me@silverwind.io> * disable preempt Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
3 years ago
Support unicode emojis and remove emojify.js (#11032) * Support unicode emojis and remove emojify.js This PR replaces all use of emojify.js and adds unicode emoji support to various areas of gitea. This works in a few ways: First it adds emoji parsing support into gitea itself. This allows us to * Render emojis from valid alias (:smile:) * Detect unicode emojis and let us put them in their own class with proper aria-labels and styling * Easily allow for custom "emoji" * Support all emoji rendering and features without javascript * Uses plain unicode and lets the system render in appropriate emoji font * Doesn't leave us relying on external sources for updates/fixes/features That same list of emoji is also used to create a json file which replaces the part of emojify.js that populates the emoji search tribute. This file is about 35KB with GZIP turned on and I've set it to load after the page renders to not hinder page load time (and this removes loading emojify.js also) For custom "emoji" it uses a pretty simple scheme of just looking for /emojis/img/name.png where name is something a user has put in the "allowed reactions" setting we already have. The gitea reaction that was previously hard coded into a forked copy of emojify.js is included and works as a custom reaction under this method. The emoji data sourced here is from https://github.com/github/gemoji which is the gem library Github uses for their emoji rendering (and a data source for other sites). So we should be able to easily render any emoji and :alias: that Github can, removing any errors from migrated content. They also update it as well, so we can sync when there are new unicode emoji lists released. I've included a slimmed down and slightly modified forked copy of https://github.com/knq/emoji to make up our own emoji module. The code is pretty straight forward and again allows us to have a lot of flexibility in what happens. I had seen a few comments about performance in some of the other threads if we render this ourselves, but there doesn't seem to be any issue here. In a test it can parse, convert, and render 1,000 emojis inside of a large markdown table in about 100ms on my laptop (which is many more emojis than will ever be in any normal issue). This also prevents any flickering and other weirdness from using javascript to render some things while using go for others. Not included here are image fall back URLS. I don't really think they are necessary for anything new being written in 2020. However, managing the emoji ourselves would allow us to add these as a feature later on if it seems necessary. Fixes: https://github.com/go-gitea/gitea/issues/9182 Fixes: https://github.com/go-gitea/gitea/issues/8974 Fixes: https://github.com/go-gitea/gitea/issues/8953 Fixes: https://github.com/go-gitea/gitea/issues/6628 Fixes: https://github.com/go-gitea/gitea/issues/5130 * add new shared function emojiHTML * don't increase emoji size in issue title * Update templates/repo/issue/view_content/add_reaction.tmpl Co-Authored-By: 6543 <6543@obermui.de> * Support for emoji rendering in various templates * Render code and review comments as they should be * Better way to handle mail subjects * insert unicode from tribute selection * Add template helper for plain text when needed * Use existing replace function I forgot about * Don't include emoji greater than Unicode Version 12 Only include emoji and aliases in JSON * Update build/generate-emoji.go * Tweak regex slightly to really match everything including random invisible characters. Run tests for every emoji we have * final updates * code review * code review * hard code gitea custom emoji to match previous behavior * Update .eslintrc Co-Authored-By: silverwind <me@silverwind.io> * disable preempt Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
3 years ago
Check commit message hashes before making links (#7713) * Check commit message hashes before making links Previously, when formatting commit messages, anything that looked like SHA1 hashes was turned into a link using regex. This meant that certain phrases or numbers such as `777777` or `deadbeef` could be recognized as a commit even if the repository has no commit with those hashes. This change will make it so that anything that looks like a SHA1 hash using regex will then also be checked to ensure that there is a commit in the repository with that hash before making a link. Signed-off-by: Gary Kim <gary@garykim.dev> * Use gogit to check if commit exists This commit modifies the commit hash check in the render for commit messages to use gogit for better performance. Signed-off-by: Gary Kim <gary@garykim.dev> * Make code cleaner Signed-off-by: Gary Kim <gary@garykim.dev> * Use rev-parse to check if commit exists Signed-off-by: Gary Kim <gary@garykim.dev> * Add and modify tests for checking hashes in html link rendering Signed-off-by: Gary Kim <gary@garykim.dev> * Return error in sha1CurrentPatternProcessor Co-Authored-By: mrsdizzie <info@mrsdizzie.com> * Import Gitea log module Signed-off-by: Gary Kim <gary@garykim.dev> * Revert "Return error in sha1CurrentPatternProcessor" This reverts commit 28f561cac46ef7e51aa26aefcbe9aca4671366a6. Signed-off-by: Gary Kim <gary@garykim.dev> * Add debug logging to sha1CurrentPatternProcessor This will log errors by the git command run in sha1CurrentPatternProcessor if the error is one that was unexpected. Signed-off-by: Gary Kim <gary@garykim.dev>
3 years ago
Check commit message hashes before making links (#7713) * Check commit message hashes before making links Previously, when formatting commit messages, anything that looked like SHA1 hashes was turned into a link using regex. This meant that certain phrases or numbers such as `777777` or `deadbeef` could be recognized as a commit even if the repository has no commit with those hashes. This change will make it so that anything that looks like a SHA1 hash using regex will then also be checked to ensure that there is a commit in the repository with that hash before making a link. Signed-off-by: Gary Kim <gary@garykim.dev> * Use gogit to check if commit exists This commit modifies the commit hash check in the render for commit messages to use gogit for better performance. Signed-off-by: Gary Kim <gary@garykim.dev> * Make code cleaner Signed-off-by: Gary Kim <gary@garykim.dev> * Use rev-parse to check if commit exists Signed-off-by: Gary Kim <gary@garykim.dev> * Add and modify tests for checking hashes in html link rendering Signed-off-by: Gary Kim <gary@garykim.dev> * Return error in sha1CurrentPatternProcessor Co-Authored-By: mrsdizzie <info@mrsdizzie.com> * Import Gitea log module Signed-off-by: Gary Kim <gary@garykim.dev> * Revert "Return error in sha1CurrentPatternProcessor" This reverts commit 28f561cac46ef7e51aa26aefcbe9aca4671366a6. Signed-off-by: Gary Kim <gary@garykim.dev> * Add debug logging to sha1CurrentPatternProcessor This will log errors by the git command run in sha1CurrentPatternProcessor if the error is one that was unexpected. Signed-off-by: Gary Kim <gary@garykim.dev>
3 years ago
  1. // Copyright 2017 The Gitea Authors. All rights reserved.
  2. // Use of this source code is governed by a MIT-style
  3. // license that can be found in the LICENSE file.
  4. package markup
  5. import (
  6. "bytes"
  7. "fmt"
  8. "net/url"
  9. "path"
  10. "path/filepath"
  11. "regexp"
  12. "strings"
  13. "code.gitea.io/gitea/modules/base"
  14. "code.gitea.io/gitea/modules/emoji"
  15. "code.gitea.io/gitea/modules/git"
  16. "code.gitea.io/gitea/modules/log"
  17. "code.gitea.io/gitea/modules/markup/common"
  18. "code.gitea.io/gitea/modules/references"
  19. "code.gitea.io/gitea/modules/setting"
  20. "code.gitea.io/gitea/modules/util"
  21. "github.com/unknwon/com"
  22. "golang.org/x/net/html"
  23. "golang.org/x/net/html/atom"
  24. "mvdan.cc/xurls/v2"
  25. )
  26. // Issue name styles
  27. const (
  28. IssueNameStyleNumeric = "numeric"
  29. IssueNameStyleAlphanumeric = "alphanumeric"
  30. )
  31. var (
  32. // NOTE: All below regex matching do not perform any extra validation.
  33. // Thus a link is produced even if the linked entity does not exist.
  34. // While fast, this is also incorrect and lead to false positives.
  35. // TODO: fix invalid linking issue
  36. // sha1CurrentPattern matches string that represents a commit SHA, e.g. d8a994ef243349f321568f9e36d5c3f444b99cae
  37. // Although SHA1 hashes are 40 chars long, the regex matches the hash from 7 to 40 chars in length
  38. // so that abbreviated hash links can be used as well. This matches git and github useability.
  39. sha1CurrentPattern = regexp.MustCompile(`(?:\s|^|\(|\[)([0-9a-f]{7,40})(?:\s|$|\)|\]|[.,](\s|$))`)
  40. // shortLinkPattern matches short but difficult to parse [[name|link|arg=test]] syntax
  41. shortLinkPattern = regexp.MustCompile(`\[\[(.*?)\]\](\w*)`)
  42. // anySHA1Pattern allows to split url containing SHA into parts
  43. anySHA1Pattern = regexp.MustCompile(`https?://(?:\S+/){4}([0-9a-f]{40})(/[^#\s]+)?(#\S+)?`)
  44. validLinksPattern = regexp.MustCompile(`^[a-z][\w-]+://`)
  45. // While this email regex is definitely not perfect and I'm sure you can come up
  46. // with edge cases, it is still accepted by the CommonMark specification, as
  47. // well as the HTML5 spec:
  48. // http://spec.commonmark.org/0.28/#email-address
  49. // https://html.spec.whatwg.org/multipage/input.html#e-mail-state-(type%3Demail)
  50. emailRegex = regexp.MustCompile("(?:\\s|^|\\(|\\[)([a-zA-Z0-9.!#$%&'*+\\/=?^_`{|}~-]+@[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?(?:\\.[a-zA-Z0-9]{2,}(?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?)+)(?:\\s|$|\\)|\\]|\\.(\\s|$))")
  51. // blackfriday extensions create IDs like fn:user-content-footnote
  52. blackfridayExtRegex = regexp.MustCompile(`[^:]*:user-content-`)
  53. // EmojiShortCodeRegex find emoji by alias like :smile:
  54. EmojiShortCodeRegex = regexp.MustCompile(`\:[\w\+\-]+\:{1}`)
  55. )
  56. // CSS class for action keywords (e.g. "closes: #1")
  57. const keywordClass = "issue-keyword"
  58. // regexp for full links to issues/pulls
  59. var issueFullPattern *regexp.Regexp
  60. // IsLink reports whether link fits valid format.
  61. func IsLink(link []byte) bool {
  62. return isLink(link)
  63. }
  64. // isLink reports whether link fits valid format.
  65. func isLink(link []byte) bool {
  66. return validLinksPattern.Match(link)
  67. }
  68. func isLinkStr(link string) bool {
  69. return validLinksPattern.MatchString(link)
  70. }
  71. func getIssueFullPattern() *regexp.Regexp {
  72. if issueFullPattern == nil {
  73. issueFullPattern = regexp.MustCompile(regexp.QuoteMeta(setting.AppURL) +
  74. `\w+/\w+/(?:issues|pulls)/((?:\w{1,10}-)?[1-9][0-9]*)([\?|#]\S+.(\S+)?)?\b`)
  75. }
  76. return issueFullPattern
  77. }
  78. // CustomLinkURLSchemes allows for additional schemes to be detected when parsing links within text
  79. func CustomLinkURLSchemes(schemes []string) {
  80. schemes = append(schemes, "http", "https")
  81. withAuth := make([]string, 0, len(schemes))
  82. validScheme := regexp.MustCompile(`^[a-z]+$`)
  83. for _, s := range schemes {
  84. if !validScheme.MatchString(s) {
  85. continue
  86. }
  87. without := false
  88. for _, sna := range xurls.SchemesNoAuthority {
  89. if s == sna {
  90. without = true
  91. break
  92. }
  93. }
  94. if without {
  95. s += ":"
  96. } else {
  97. s += "://"
  98. }
  99. withAuth = append(withAuth, s)
  100. }
  101. common.LinkRegex, _ = xurls.StrictMatchingScheme(strings.Join(withAuth, "|"))
  102. }
  103. // IsSameDomain checks if given url string has the same hostname as current Gitea instance
  104. func IsSameDomain(s string) bool {
  105. if strings.HasPrefix(s, "/") {
  106. return true
  107. }
  108. if uapp, err := url.Parse(setting.AppURL); err == nil {
  109. if u, err := url.Parse(s); err == nil {
  110. return u.Host == uapp.Host
  111. }
  112. return false
  113. }
  114. return false
  115. }
  116. type postProcessError struct {
  117. context string
  118. err error
  119. }
  120. func (p *postProcessError) Error() string {
  121. return "PostProcess: " + p.context + ", " + p.err.Error()
  122. }
  123. type processor func(ctx *postProcessCtx, node *html.Node)
  124. var defaultProcessors = []processor{
  125. fullIssuePatternProcessor,
  126. fullSha1PatternProcessor,
  127. shortLinkProcessor,
  128. linkProcessor,
  129. mentionProcessor,
  130. issueIndexPatternProcessor,
  131. sha1CurrentPatternProcessor,
  132. emailAddressProcessor,
  133. emojiProcessor,
  134. emojiShortCodeProcessor,
  135. }
  136. type postProcessCtx struct {
  137. metas map[string]string
  138. urlPrefix string
  139. isWikiMarkdown bool
  140. // processors used by this context.
  141. procs []processor
  142. }
  143. // PostProcess does the final required transformations to the passed raw HTML
  144. // data, and ensures its validity. Transformations include: replacing links and
  145. // emails with HTML links, parsing shortlinks in the format of [[Link]], like
  146. // MediaWiki, linking issues in the format #ID, and mentions in the format
  147. // @user, and others.
  148. func PostProcess(
  149. rawHTML []byte,
  150. urlPrefix string,
  151. metas map[string]string,
  152. isWikiMarkdown bool,
  153. ) ([]byte, error) {
  154. // create the context from the parameters
  155. ctx := &postProcessCtx{
  156. metas: metas,
  157. urlPrefix: urlPrefix,
  158. isWikiMarkdown: isWikiMarkdown,
  159. procs: defaultProcessors,
  160. }
  161. return ctx.postProcess(rawHTML)
  162. }
  163. var commitMessageProcessors = []processor{
  164. fullIssuePatternProcessor,
  165. fullSha1PatternProcessor,
  166. linkProcessor,
  167. mentionProcessor,
  168. issueIndexPatternProcessor,
  169. sha1CurrentPatternProcessor,
  170. emailAddressProcessor,
  171. emojiProcessor,
  172. emojiShortCodeProcessor,
  173. }
  174. // RenderCommitMessage will use the same logic as PostProcess, but will disable
  175. // the shortLinkProcessor and will add a defaultLinkProcessor if defaultLink is
  176. // set, which changes every text node into a link to the passed default link.
  177. func RenderCommitMessage(
  178. rawHTML []byte,
  179. urlPrefix, defaultLink string,
  180. metas map[string]string,
  181. ) ([]byte, error) {
  182. ctx := &postProcessCtx{
  183. metas: metas,
  184. urlPrefix: urlPrefix,
  185. procs: commitMessageProcessors,
  186. }
  187. if defaultLink != "" {
  188. // we don't have to fear data races, because being
  189. // commitMessageProcessors of fixed len and cap, every time we append
  190. // something to it the slice is realloc+copied, so append always
  191. // generates the slice ex-novo.
  192. ctx.procs = append(ctx.procs, genDefaultLinkProcessor(defaultLink))
  193. }
  194. return ctx.postProcess(rawHTML)
  195. }
  196. var commitMessageSubjectProcessors = []processor{
  197. fullIssuePatternProcessor,
  198. fullSha1PatternProcessor,
  199. linkProcessor,
  200. mentionProcessor,
  201. issueIndexPatternProcessor,
  202. sha1CurrentPatternProcessor,
  203. emojiShortCodeProcessor,
  204. emojiProcessor,
  205. }
  206. var emojiProcessors = []processor{
  207. emojiShortCodeProcessor,
  208. emojiProcessor,
  209. }
  210. // RenderCommitMessageSubject will use the same logic as PostProcess and
  211. // RenderCommitMessage, but will disable the shortLinkProcessor and
  212. // emailAddressProcessor, will add a defaultLinkProcessor if defaultLink is set,
  213. // which changes every text node into a link to the passed default link.
  214. func RenderCommitMessageSubject(
  215. rawHTML []byte,
  216. urlPrefix, defaultLink string,
  217. metas map[string]string,
  218. ) ([]byte, error) {
  219. ctx := &postProcessCtx{
  220. metas: metas,
  221. urlPrefix: urlPrefix,
  222. procs: commitMessageSubjectProcessors,
  223. }
  224. if defaultLink != "" {
  225. // we don't have to fear data races, because being
  226. // commitMessageSubjectProcessors of fixed len and cap, every time we
  227. // append something to it the slice is realloc+copied, so append always
  228. // generates the slice ex-novo.
  229. ctx.procs = append(ctx.procs, genDefaultLinkProcessor(defaultLink))
  230. }
  231. return ctx.postProcess(rawHTML)
  232. }
  233. // RenderIssueTitle to process title on individual issue/pull page
  234. func RenderIssueTitle(
  235. rawHTML []byte,
  236. urlPrefix string,
  237. metas map[string]string,
  238. ) ([]byte, error) {
  239. ctx := &postProcessCtx{
  240. metas: metas,
  241. urlPrefix: urlPrefix,
  242. procs: []processor{
  243. issueIndexPatternProcessor,
  244. sha1CurrentPatternProcessor,
  245. emojiShortCodeProcessor,
  246. emojiProcessor,
  247. },
  248. }
  249. return ctx.postProcess(rawHTML)
  250. }
  251. // RenderDescriptionHTML will use similar logic as PostProcess, but will
  252. // use a single special linkProcessor.
  253. func RenderDescriptionHTML(
  254. rawHTML []byte,
  255. urlPrefix string,
  256. metas map[string]string,
  257. ) ([]byte, error) {
  258. ctx := &postProcessCtx{
  259. metas: metas,
  260. urlPrefix: urlPrefix,
  261. procs: []processor{
  262. descriptionLinkProcessor,
  263. emojiShortCodeProcessor,
  264. emojiProcessor,
  265. },
  266. }
  267. return ctx.postProcess(rawHTML)
  268. }
  269. // RenderEmoji for when we want to just process emoji and shortcodes
  270. // in various places it isn't already run through the normal markdown procesor
  271. func RenderEmoji(
  272. rawHTML []byte,
  273. ) ([]byte, error) {
  274. ctx := &postProcessCtx{
  275. procs: emojiProcessors,
  276. }
  277. return ctx.postProcess(rawHTML)
  278. }
  279. var tagCleaner = regexp.MustCompile(`<((?:/?\w+/\w+)|(?:/[\w ]+/)|(/?[hH][tT][mM][lL][ />])|(/?[hH][eE][aA][dD][ />]))`)
  280. var nulCleaner = strings.NewReplacer("\000", "")
  281. func (ctx *postProcessCtx) postProcess(rawHTML []byte) ([]byte, error) {
  282. if ctx.procs == nil {
  283. ctx.procs = defaultProcessors
  284. }
  285. // give a generous extra 50 bytes
  286. res := bytes.NewBuffer(make([]byte, 0, len(rawHTML)+50))
  287. // prepend "<html><body>"
  288. _, _ = res.WriteString("<html><body>")
  289. // Strip out nuls - they're always invalid
  290. _, _ = res.Write(tagCleaner.ReplaceAll([]byte(nulCleaner.Replace(string(rawHTML))), []byte("&lt;$1")))
  291. // close the tags
  292. _, _ = res.WriteString("</body></html>")
  293. // parse the HTML
  294. nodes, err := html.ParseFragment(res, nil)
  295. if err != nil {
  296. return nil, &postProcessError{"invalid HTML", err}
  297. }
  298. for _, node := range nodes {
  299. ctx.visitNode(node, true)
  300. }
  301. newNodes := make([]*html.Node, 0, len(nodes))
  302. for _, node := range nodes {
  303. if node.Data == "html" {
  304. node = node.FirstChild
  305. for node != nil && node.Data != "body" {
  306. node = node.NextSibling
  307. }
  308. }
  309. if node == nil {
  310. continue
  311. }
  312. if node.Data == "body" {
  313. child := node.FirstChild
  314. for child != nil {
  315. newNodes = append(newNodes, child)
  316. child = child.NextSibling
  317. }
  318. } else {
  319. newNodes = append(newNodes, node)
  320. }
  321. }
  322. nodes = newNodes
  323. // Create buffer in which the data will be placed again. We know that the
  324. // length will be at least that of res; to spare a few alloc+copy, we
  325. // reuse res, resetting its length to 0.
  326. res.Reset()
  327. // Render everything to buf.
  328. for _, node := range nodes {
  329. err = html.Render(res, node)
  330. if err != nil {
  331. return nil, &postProcessError{"error rendering processed HTML", err}
  332. }
  333. }
  334. // Everything done successfully, return parsed data.
  335. return res.Bytes(), nil
  336. }
  337. func (ctx *postProcessCtx) visitNode(node *html.Node, visitText bool) {
  338. // Add user-content- to IDs if they don't already have them
  339. for idx, attr := range node.Attr {
  340. if attr.Key == "id" && !(strings.HasPrefix(attr.Val, "user-content-") || blackfridayExtRegex.MatchString(attr.Val)) {
  341. node.Attr[idx].Val = "user-content-" + attr.Val
  342. }
  343. if attr.Key == "class" && attr.Val == "emoji" {
  344. visitText = false
  345. }
  346. }
  347. // We ignore code, pre and already generated links.
  348. switch node.Type {
  349. case html.TextNode:
  350. if visitText {
  351. ctx.textNode(node)
  352. }
  353. case html.ElementNode:
  354. if node.Data == "img" {
  355. attrs := node.Attr
  356. for idx, attr := range attrs {
  357. if attr.Key != "src" {
  358. continue
  359. }
  360. link := []byte(attr.Val)
  361. if len(link) > 0 && !IsLink(link) {
  362. prefix := ctx.urlPrefix
  363. if ctx.isWikiMarkdown {
  364. prefix = util.URLJoin(prefix, "wiki", "raw")
  365. }
  366. prefix = strings.Replace(prefix, "/src/", "/media/", 1)
  367. lnk := string(link)
  368. lnk = util.URLJoin(prefix, lnk)
  369. link = []byte(lnk)
  370. }
  371. node.Attr[idx].Val = string(link)
  372. }
  373. } else if node.Data == "a" {
  374. visitText = false
  375. } else if node.Data == "code" || node.Data == "pre" {
  376. return
  377. } else if node.Data == "i" {
  378. for _, attr := range node.Attr {
  379. if attr.Key != "class" {
  380. continue
  381. }
  382. classes := strings.Split(attr.Val, " ")
  383. for i, class := range classes {
  384. if class == "icon" {
  385. classes[0], classes[i] = classes[i], classes[0]
  386. attr.Val = strings.Join(classes, " ")
  387. // Remove all children of icons
  388. child := node.FirstChild
  389. for child != nil {
  390. node.RemoveChild(child)
  391. child = node.FirstChild
  392. }
  393. break
  394. }
  395. }
  396. }
  397. }
  398. for n := node.FirstChild; n != nil; n = n.NextSibling {
  399. ctx.visitNode(n, visitText)
  400. }
  401. }
  402. // ignore everything else
  403. }
  404. // textNode runs the passed node through various processors, in order to handle
  405. // all kinds of special links handled by the post-processing.
  406. func (ctx *postProcessCtx) textNode(node *html.Node) {
  407. for _, processor := range ctx.procs {
  408. processor(ctx, node)
  409. }
  410. }
  411. // createKeyword() renders a highlighted version of an action keyword
  412. func createKeyword(content string) *html.Node {
  413. span := &html.Node{
  414. Type: html.ElementNode,
  415. Data: atom.Span.String(),
  416. Attr: []html.Attribute{},
  417. }
  418. span.Attr = append(span.Attr, html.Attribute{Key: "class", Val: keywordClass})
  419. text := &html.Node{
  420. Type: html.TextNode,
  421. Data: content,
  422. }
  423. span.AppendChild(text)
  424. return span
  425. }
  426. func createEmoji(content, class, name string) *html.Node {
  427. span := &html.Node{
  428. Type: html.ElementNode,
  429. Data: atom.Span.String(),
  430. Attr: []html.Attribute{},
  431. }
  432. if class != "" {
  433. span.Attr = append(span.Attr, html.Attribute{Key: "class", Val: class})
  434. }
  435. if name != "" {
  436. span.Attr = append(span.Attr, html.Attribute{Key: "aria-label", Val: name})
  437. }
  438. text := &html.Node{
  439. Type: html.TextNode,
  440. Data: content,
  441. }
  442. span.AppendChild(text)
  443. return span
  444. }
  445. func createCustomEmoji(alias, class string) *html.Node {
  446. span := &html.Node{
  447. Type: html.ElementNode,
  448. Data: atom.Span.String(),
  449. Attr: []html.Attribute{},
  450. }
  451. if class != "" {
  452. span.Attr = append(span.Attr, html.Attribute{Key: "class", Val: class})
  453. span.Attr = append(span.Attr, html.Attribute{Key: "aria-label", Val: alias})
  454. }
  455. img := &html.Node{
  456. Type: html.ElementNode,
  457. DataAtom: atom.Img,
  458. Data: "img",
  459. Attr: []html.Attribute{},
  460. }
  461. if class != "" {
  462. img.Attr = append(img.Attr, html.Attribute{Key: "alt", Val: fmt.Sprintf(`:%s:`, alias)})
  463. img.Attr = append(img.Attr, html.Attribute{Key: "src", Val: fmt.Sprintf(`%s/img/emoji/%s.png`, setting.StaticURLPrefix, alias)})
  464. }
  465. span.AppendChild(img)
  466. return span
  467. }
  468. func createLink(href, content, class string) *html.Node {
  469. a := &html.Node{
  470. Type: html.ElementNode,
  471. Data: atom.A.String(),
  472. Attr: []html.Attribute{{Key: "href", Val: href}},
  473. }
  474. if class != "" {
  475. a.Attr = append(a.Attr, html.Attribute{Key: "class", Val: class})
  476. }
  477. text := &html.Node{
  478. Type: html.TextNode,
  479. Data: content,
  480. }
  481. a.AppendChild(text)
  482. return a
  483. }
  484. func createCodeLink(href, content, class string) *html.Node {
  485. a := &html.Node{
  486. Type: html.ElementNode,
  487. Data: atom.A.String(),
  488. Attr: []html.Attribute{{Key: "href", Val: href}},
  489. }
  490. if class != "" {
  491. a.Attr = append(a.Attr, html.Attribute{Key: "class", Val: class})
  492. }
  493. text := &html.Node{
  494. Type: html.TextNode,
  495. Data: content,
  496. }
  497. code := &html.Node{
  498. Type: html.ElementNode,
  499. Data: atom.Code.String(),
  500. Attr: []html.Attribute{{Key: "class", Val: "nohighlight"}},
  501. }
  502. code.AppendChild(text)
  503. a.AppendChild(code)
  504. return a
  505. }
  506. // replaceContent takes text node, and in its content it replaces a section of
  507. // it with the specified newNode.
  508. func replaceContent(node *html.Node, i, j int, newNode *html.Node) {
  509. replaceContentList(node, i, j, []*html.Node{newNode})
  510. }
  511. // replaceContentList takes text node, and in its content it replaces a section of
  512. // it with the specified newNodes. An example to visualize how this can work can
  513. // be found here: https://play.golang.org/p/5zP8NnHZ03s
  514. func replaceContentList(node *html.Node, i, j int, newNodes []*html.Node) {
  515. // get the data before and after the match
  516. before := node.Data[:i]
  517. after := node.Data[j:]
  518. // Replace in the current node the text, so that it is only what it is
  519. // supposed to have.
  520. node.Data = before
  521. // Get the current next sibling, before which we place the replaced data,
  522. // and after that we place the new text node.
  523. nextSibling := node.NextSibling
  524. for _, n := range newNodes {
  525. node.Parent.InsertBefore(n, nextSibling)
  526. }
  527. if after != "" {
  528. node.Parent.InsertBefore(&html.Node{
  529. Type: html.TextNode,
  530. Data: after,
  531. }, nextSibling)
  532. }
  533. }
  534. func mentionProcessor(ctx *postProcessCtx, node *html.Node) {
  535. // We replace only the first mention; other mentions will be addressed later
  536. found, loc := references.FindFirstMentionBytes([]byte(node.Data))
  537. if !found {
  538. return
  539. }
  540. mention := node.Data[loc.Start:loc.End]
  541. var teams string
  542. teams, ok := ctx.metas["teams"]
  543. // FIXME: util.URLJoin may not be necessary here:
  544. // - setting.AppURL is defined to have a terminal '/' so unless mention[1:]
  545. // is an AppSubURL link we can probably fallback to concatenation.
  546. // team mention should follow @orgName/teamName style
  547. if ok && strings.Contains(mention, "/") {
  548. mentionOrgAndTeam := strings.Split(mention, "/")
  549. if mentionOrgAndTeam[0][1:] == ctx.metas["org"] && strings.Contains(teams, ","+strings.ToLower(mentionOrgAndTeam[1])+",") {
  550. replaceContent(node, loc.Start, loc.End, createLink(util.URLJoin(setting.AppURL, "org", ctx.metas["org"], "teams", mentionOrgAndTeam[1]), mention, "mention"))
  551. }
  552. return
  553. }
  554. replaceContent(node, loc.Start, loc.End, createLink(util.URLJoin(setting.AppURL, mention[1:]), mention, "mention"))
  555. }
  556. func shortLinkProcessor(ctx *postProcessCtx, node *html.Node) {
  557. shortLinkProcessorFull(ctx, node, false)
  558. }
  559. func shortLinkProcessorFull(ctx *postProcessCtx, node *html.Node, noLink bool) {
  560. m := shortLinkPattern.FindStringSubmatchIndex(node.Data)
  561. if m == nil {
  562. return
  563. }
  564. content := node.Data[m[2]:m[3]]
  565. tail := node.Data[m[4]:m[5]]
  566. props := make(map[string]string)
  567. // MediaWiki uses [[link|text]], while GitHub uses [[text|link]]
  568. // It makes page handling terrible, but we prefer GitHub syntax
  569. // And fall back to MediaWiki only when it is obvious from the look
  570. // Of text and link contents
  571. sl := strings.Split(content, "|")
  572. for _, v := range sl {
  573. if equalPos := strings.IndexByte(v, '='); equalPos == -1 {
  574. // There is no equal in this argument; this is a mandatory arg
  575. if props["name"] == "" {
  576. if isLinkStr(v) {
  577. // If we clearly see it is a link, we save it so
  578. // But first we need to ensure, that if both mandatory args provided
  579. // look like links, we stick to GitHub syntax
  580. if props["link"] != "" {
  581. props["name"] = props["link"]
  582. }
  583. props["link"] = strings.TrimSpace(v)
  584. } else {
  585. props["name"] = v
  586. }
  587. } else {
  588. props["link"] = strings.TrimSpace(v)
  589. }
  590. } else {
  591. // There is an equal; optional argument.
  592. sep := strings.IndexByte(v, '=')
  593. key, val := v[:sep], html.UnescapeString(v[sep+1:])
  594. // When parsing HTML, x/net/html will change all quotes which are
  595. // not used for syntax into UTF-8 quotes. So checking val[0] won't
  596. // be enough, since that only checks a single byte.
  597. if len(val) > 1 {
  598. if (strings.HasPrefix(val, "ā€œ") && strings.HasSuffix(val, "ā€")) ||
  599. (strings.HasPrefix(val, "ā€˜") && strings.HasSuffix(val, "ā€™")) {
  600. const lenQuote = len("ā€˜")
  601. val = val[lenQuote : len(val)-lenQuote]
  602. } else if (strings.HasPrefix(val, "\"") && strings.HasSuffix(val, "\"")) ||
  603. (strings.HasPrefix(val, "'") && strings.HasSuffix(val, "'")) {
  604. val = val[1 : len(val)-1]
  605. } else if strings.HasPrefix(val, "'") && strings.HasSuffix(val, "ā€™") {
  606. const lenQuote = len("ā€˜")
  607. val = val[1 : len(val)-lenQuote]
  608. }
  609. }
  610. props[key] = val
  611. }
  612. }
  613. var name, link string
  614. if props["link"] != "" {
  615. link = props["link"]
  616. } else if props["name"] != "" {
  617. link = props["name"]
  618. }
  619. if props["title"] != "" {
  620. name = props["title"]
  621. } else if props["name"] != "" {
  622. name = props["name"]
  623. } else {
  624. name = link
  625. }
  626. name += tail
  627. image := false
  628. switch ext := filepath.Ext(link); ext {
  629. // fast path: empty string, ignore
  630. case "":
  631. break
  632. case ".jpg", ".jpeg", ".png", ".tif", ".tiff", ".webp", ".gif", ".bmp", ".ico", ".svg":
  633. image = true
  634. }
  635. childNode := &html.Node{}
  636. linkNode := &html.Node{
  637. FirstChild: childNode,
  638. LastChild: childNode,
  639. Type: html.ElementNode,
  640. Data: "a",
  641. DataAtom: atom.A,
  642. }
  643. childNode.Parent = linkNode
  644. absoluteLink := isLinkStr(link)
  645. if !absoluteLink {
  646. if image {
  647. link = strings.ReplaceAll(link, " ", "+")
  648. } else {
  649. link = strings.ReplaceAll(link, " ", "-")
  650. }
  651. if !strings.Contains(link, "/") {
  652. link = url.PathEscape(link)
  653. }
  654. }
  655. urlPrefix := ctx.urlPrefix
  656. if image {
  657. if !absoluteLink {
  658. if IsSameDomain(urlPrefix) {
  659. urlPrefix = strings.Replace(urlPrefix, "/src/", "/raw/", 1)
  660. }
  661. if ctx.isWikiMarkdown {
  662. link = util.URLJoin("wiki", "raw", link)
  663. }
  664. link = util.URLJoin(urlPrefix, link)
  665. }
  666. title := props["title"]
  667. if title == "" {
  668. title = props["alt"]
  669. }
  670. if title == "" {
  671. title = path.Base(name)
  672. }
  673. alt := props["alt"]
  674. if alt == "" {
  675. alt = name
  676. }
  677. // make the childNode an image - if we can, we also place the alt
  678. childNode.Type = html.ElementNode
  679. childNode.Data = "img"
  680. childNode.DataAtom = atom.Img
  681. childNode.Attr = []html.Attribute{
  682. {Key: "src", Val: link},
  683. {Key: "title", Val: title},
  684. {Key: "alt", Val: alt},
  685. }
  686. if alt == "" {
  687. childNode.Attr = childNode.Attr[:2]
  688. }
  689. } else {
  690. if !absoluteLink {
  691. if ctx.isWikiMarkdown {
  692. link = util.URLJoin("wiki", link)
  693. }
  694. link = util.URLJoin(urlPrefix, link)
  695. }
  696. childNode.Type = html.TextNode
  697. childNode.Data = name
  698. }
  699. if noLink {
  700. linkNode = childNode
  701. } else {
  702. linkNode.Attr = []html.Attribute{{Key: "href", Val: link}}
  703. }
  704. replaceContent(node, m[0], m[1], linkNode)
  705. }
  706. func fullIssuePatternProcessor(ctx *postProcessCtx, node *html.Node) {
  707. if ctx.metas == nil {
  708. return
  709. }
  710. m := getIssueFullPattern().FindStringSubmatchIndex(node.Data)
  711. if m == nil {
  712. return
  713. }
  714. link := node.Data[m[0]:m[1]]
  715. id := "#" + node.Data[m[2]:m[3]]
  716. // extract repo and org name from matched link like
  717. // http://localhost:3000/gituser/myrepo/issues/1
  718. linkParts := strings.Split(path.Clean(link), "/")
  719. matchOrg := linkParts[len(linkParts)-4]
  720. matchRepo := linkParts[len(linkParts)-3]
  721. if matchOrg == ctx.metas["user"] && matchRepo == ctx.metas["repo"] {
  722. // TODO if m[4]:m[5] is not nil, then link is to a comment,
  723. // and we should indicate that in the text somehow
  724. replaceContent(node, m[0], m[1], createLink(link, id, "ref-issue"))
  725. } else {
  726. orgRepoID := matchOrg + "/" + matchRepo + id
  727. replaceContent(node, m[0], m[1], createLink(link, orgRepoID, "ref-issue"))
  728. }
  729. }
  730. func issueIndexPatternProcessor(ctx *postProcessCtx, node *html.Node) {
  731. if ctx.metas == nil {
  732. return
  733. }
  734. var (
  735. found bool
  736. ref *references.RenderizableReference
  737. )
  738. _, exttrack := ctx.metas["format"]
  739. alphanum := ctx.metas["style"] == IssueNameStyleAlphanumeric
  740. // Repos with external issue trackers might still need to reference local PRs
  741. // We need to concern with the first one that shows up in the text, whichever it is
  742. found, ref = references.FindRenderizableReferenceNumeric(node.Data, exttrack && alphanum)
  743. if exttrack && alphanum {
  744. if found2, ref2 := references.FindRenderizableReferenceAlphanumeric(node.Data); found2 {
  745. if !found || ref2.RefLocation.Start < ref.RefLocation.Start {
  746. found = true
  747. ref = ref2
  748. }
  749. }
  750. }
  751. if !found {
  752. return
  753. }
  754. var link *html.Node
  755. reftext := node.Data[ref.RefLocation.Start:ref.RefLocation.End]
  756. if exttrack && !ref.IsPull {
  757. ctx.metas["index"] = ref.Issue
  758. link = createLink(com.Expand(ctx.metas["format"], ctx.metas), reftext, "ref-issue")
  759. } else {
  760. // Path determines the type of link that will be rendered. It's unknown at this point whether
  761. // the linked item is actually a PR or an issue. Luckily it's of no real consequence because
  762. // Gitea will redirect on click as appropriate.
  763. path := "issues"
  764. if ref.IsPull {
  765. path = "pulls"
  766. }
  767. if ref.Owner == "" {
  768. link = createLink(util.URLJoin(setting.AppURL, ctx.metas["user"], ctx.metas["repo"], path, ref.Issue), reftext, "ref-issue")
  769. } else {
  770. link = createLink(util.URLJoin(setting.AppURL, ref.Owner, ref.Name, path, ref.Issue), reftext, "ref-issue")
  771. }
  772. }
  773. if ref.Action == references.XRefActionNone {
  774. replaceContent(node, ref.RefLocation.Start, ref.RefLocation.End, link)
  775. return
  776. }
  777. // Decorate action keywords if actionable
  778. var keyword *html.Node
  779. if references.IsXrefActionable(ref, exttrack, alphanum) {
  780. keyword = createKeyword(node.Data[ref.ActionLocation.Start:ref.ActionLocation.End])
  781. } else {
  782. keyword = &html.Node{
  783. Type: html.TextNode,
  784. Data: node.Data[ref.ActionLocation.Start:ref.ActionLocation.End],
  785. }
  786. }
  787. spaces := &html.Node{
  788. Type: html.TextNode,
  789. Data: node.Data[ref.ActionLocation.End:ref.RefLocation.Start],
  790. }
  791. replaceContentList(node, ref.ActionLocation.Start, ref.RefLocation.End, []*html.Node{keyword, spaces, link})
  792. }
  793. // fullSha1PatternProcessor renders SHA containing URLs
  794. func fullSha1PatternProcessor(ctx *postProcessCtx, node *html.Node) {
  795. if ctx.metas == nil {
  796. return
  797. }
  798. m := anySHA1Pattern.FindStringSubmatchIndex(node.Data)
  799. if m == nil {
  800. return
  801. }
  802. urlFull := node.Data[m[0]:m[1]]
  803. text := base.ShortSha(node.Data[m[2]:m[3]])
  804. // 3rd capture group matches a optional path
  805. subpath := ""
  806. if m[5] > 0 {
  807. subpath = node.Data[m[4]:m[5]]
  808. }
  809. // 4th capture group matches a optional url hash
  810. hash := ""
  811. if m[7] > 0 {
  812. hash = node.Data[m[6]:m[7]][1:]
  813. }
  814. start := m[0]
  815. end := m[1]
  816. // If url ends in '.', it's very likely that it is not part of the
  817. // actual url but used to finish a sentence.
  818. if strings.HasSuffix(urlFull, ".") {
  819. end--
  820. urlFull = urlFull[:len(urlFull)-1]
  821. if hash != "" {
  822. hash = hash[:len(hash)-1]
  823. } else if subpath != "" {
  824. subpath = subpath[:len(subpath)-1]
  825. }
  826. }
  827. if subpath != "" {
  828. text += subpath
  829. }
  830. if hash != "" {
  831. text += " (" + hash + ")"
  832. }
  833. replaceContent(node, start, end, createCodeLink(urlFull, text, "commit"))
  834. }
  835. // emojiShortCodeProcessor for rendering text like :smile: into emoji
  836. func emojiShortCodeProcessor(ctx *postProcessCtx, node *html.Node) {
  837. m := EmojiShortCodeRegex.FindStringSubmatchIndex(node.Data)
  838. if m == nil {
  839. return
  840. }
  841. alias := node.Data[m[0]:m[1]]
  842. alias = strings.ReplaceAll(alias, ":", "")
  843. converted := emoji.FromAlias(alias)
  844. if converted == nil {
  845. // check if this is a custom reaction
  846. s := strings.Join(setting.UI.Reactions, " ") + "gitea"
  847. if strings.Contains(s, alias) {
  848. replaceContent(node, m[0], m[1], createCustomEmoji(alias, "emoji"))
  849. return
  850. }
  851. return
  852. }
  853. replaceContent(node, m[0], m[1], createEmoji(converted.Emoji, "emoji", converted.Description))
  854. }
  855. // emoji processor to match emoji and add emoji class
  856. func emojiProcessor(ctx *postProcessCtx, node *html.Node) {
  857. m := emoji.FindEmojiSubmatchIndex(node.Data)
  858. if m == nil {
  859. return
  860. }
  861. codepoint := node.Data[m[0]:m[1]]
  862. val := emoji.FromCode(codepoint)
  863. if val != nil {
  864. replaceContent(node, m[0], m[1], createEmoji(codepoint, "emoji", val.Description))
  865. }
  866. }
  867. // sha1CurrentPatternProcessor renders SHA1 strings to corresponding links that
  868. // are assumed to be in the same repository.
  869. func sha1CurrentPatternProcessor(ctx *postProcessCtx, node *html.Node) {
  870. if ctx.metas == nil || ctx.metas["user"] == "" || ctx.metas["repo"] == "" || ctx.metas["repoPath"] == "" {
  871. return
  872. }
  873. m := sha1CurrentPattern.FindStringSubmatchIndex(node.Data)
  874. if m == nil {
  875. return
  876. }
  877. hash := node.Data[m[2]:m[3]]
  878. // The regex does not lie, it matches the hash pattern.
  879. // However, a regex cannot know if a hash actually exists or not.
  880. // We could assume that a SHA1 hash should probably contain alphas AND numerics
  881. // but that is not always the case.
  882. // Although unlikely, deadbeef and 1234567 are valid short forms of SHA1 hash
  883. // as used by git and github for linking and thus we have to do similar.
  884. // Because of this, we check to make sure that a matched hash is actually
  885. // a commit in the repository before making it a link.
  886. if _, err := git.NewCommand("rev-parse", "--verify", hash).RunInDirBytes(ctx.metas["repoPath"]); err != nil {
  887. if !strings.Contains(err.Error(), "fatal: Needed a single revision") {
  888. log.Debug("sha1CurrentPatternProcessor git rev-parse: %v", err)
  889. }
  890. return
  891. }
  892. replaceContent(node, m[2], m[3],
  893. createCodeLink(util.URLJoin(setting.AppURL, ctx.metas["user"], ctx.metas["repo"], "commit", hash), base.ShortSha(hash), "commit"))
  894. }
  895. // emailAddressProcessor replaces raw email addresses with a mailto: link.
  896. func emailAddressProcessor(ctx *postProcessCtx, node *html.Node) {
  897. m := emailRegex.FindStringSubmatchIndex(node.Data)
  898. if m == nil {
  899. return
  900. }
  901. mail := node.Data[m[2]:m[3]]
  902. replaceContent(node, m[2], m[3], createLink("mailto:"+mail, mail, "mailto"))
  903. }
  904. // linkProcessor creates links for any HTTP or HTTPS URL not captured by
  905. // markdown.
  906. func linkProcessor(ctx *postProcessCtx, node *html.Node) {
  907. m := common.LinkRegex.FindStringIndex(node.Data)
  908. if m == nil {
  909. return
  910. }
  911. uri := node.Data[m[0]:m[1]]
  912. replaceContent(node, m[0], m[1], createLink(uri, uri, "link"))
  913. }
  914. func genDefaultLinkProcessor(defaultLink string) processor {
  915. return func(ctx *postProcessCtx, node *html.Node) {
  916. ch := &html.Node{
  917. Parent: node,
  918. Type: html.TextNode,
  919. Data: node.Data,
  920. }
  921. node.Type = html.ElementNode
  922. node.Data = "a"
  923. node.DataAtom = atom.A
  924. node.Attr = []html.Attribute{
  925. {Key: "href", Val: defaultLink},
  926. {Key: "class", Val: "default-link"},
  927. }
  928. node.FirstChild, node.LastChild = ch, ch
  929. }
  930. }
  931. // descriptionLinkProcessor creates links for DescriptionHTML
  932. func descriptionLinkProcessor(ctx *postProcessCtx, node *html.Node) {
  933. m := common.LinkRegex.FindStringIndex(node.Data)
  934. if m == nil {
  935. return
  936. }
  937. uri := node.Data[m[0]:m[1]]
  938. replaceContent(node, m[0], m[1], createDescriptionLink(uri, uri))
  939. }
  940. func createDescriptionLink(href, content string) *html.Node {
  941. textNode := &html.Node{
  942. Type: html.TextNode,
  943. Data: content,
  944. }
  945. linkNode := &html.Node{
  946. FirstChild: textNode,
  947. LastChild: textNode,
  948. Type: html.ElementNode,
  949. Data: "a",
  950. DataAtom: atom.A,
  951. Attr: []html.Attribute{
  952. {Key: "href", Val: href},
  953. {Key: "target", Val: "_blank"},
  954. {Key: "rel", Val: "noopener noreferrer"},
  955. },
  956. }
  957. textNode.Parent = linkNode
  958. return linkNode
  959. }