minor optimizations to reward system backend and UI

REWARDS.md

@@ -35,7 +35,7 @@ It should be noted that the system survey generation requires root for many of i
 
 ### Log & Survey Collection
 
-The log collection and [reward processing](#reward-processing) happens hourly.
+The log collection and [reward processing](#reward-processing) happens hourly via [skywire-reward.service](/scripts/rewards/services/skywire-reward.service) - triggered to run hourly by [skywire-reward.timer](/scripts/rewards/services/skywire-reward.timer).
 
 The log collection run can be viewed here:
 https://fiber.skywire.dev/log-collection
@@ -52,11 +52,16 @@ These `survey_whitelist` keys are specified by the [conf service](https://conf.s
 
 The collected surveys are then checked and backed up.
 
-The collection of surveys and the reward calculation happens hourly - as configured by the systemd service and timer
+The following scripts are used by the reward system:
+
+[`getlogs.sh`](/scripts/rewards/getlogs.sh) - a wrapper script for survey and transport bandwidth log collection via `skywire cli log`
+[`reward.sh`](/scripts/rewards/reward.sh) - a wrapper script for reward calculation via `skywire cli rewards`
+[`gettps.sh`](/scripts/rewards/gettps.sh) - a wrapper script for collecting responses to transport setup-node requests via `skywire svc tps ls`
+[`testproxies.sh`](/scripts/rewards/testproxies.sh) - WIP - a wrapper script for testing curl response time over the skywire socks5 proxy (not used for reward calculation)
 
 ### Reward Processing
 
-The rewards are calculated by `skywire cli rewards calc` with the aid of [reward.sh](scripts/rewards/reward.sh) to produce the reward distribution data for the previous day's uptime.
+The rewards are calculated by `skywire cli rewards calc` with the aid of [`reward.sh`](scripts/rewards/reward.sh) to produce the reward distribution data for the previous day's uptime.
 
 ### Per-IP reward limit
 
@@ -70,53 +75,27 @@ To avoid a user running multiple instances of skywire on virtual machines, the M
 
 ## Automation via systemd service
 
-Automation of the hourly log & survey collection is accomplished via systemd service and timer executing [getlogs.sh](scripts/rewards/getlogs.sh) and [reward.sh](scripts/rewards/reward.sh)
-
-/etc/systemd/system/skywire-reward.service
-```
-[Unit]
-Description=skywire reward service
-After=network.target
-
-[Service]
-Type=simple
-User=user
-Group=user
-WorkingDirectory=/path/to/github.com/reward/rewards
-ExecStart=/usr/bin/bash -c './getlogs.sh && ./reward.sh'
-
-[Install]
-WantedBy=multi-user.target
+Automation of the hourly log & survey collection is accomplished via systemd service and timer
 
-```
+/etc/systemd/system/[`skywire-reward.service`](/scripts/rewards/services/skywire-reward.service)
 
 **Note: change the user and working directory in the above systemd service**
 
-This service is called by a timer
+This service is called by a timer which triggers it to run hourly
 
-/etc/systemd/system/skywire-reward.timer
-```
-[Unit]
-Description=skywire reward timer
-After=network.target
-
-[Timer]
-OnUnitActiveSec=1h
-Unit=skywire-reward.service
-
-[Install]
-WantedBy=multi-user.target
-
-```
+/etc/systemd/system/[`skywire-reward.timer`](/scripts/rewards/services/skywire-reward.timer).
 
 
 ## fiber.skywire.dev
 
-The 'frontend' of the reward system, is currently running at [fiber.skywire.dev](https://fiber.skywire.dev) and is reliant upon on the output of certain cli commands and some scripts
+The 'frontend' of the reward system, is currently running at [fiber.skywire.dev](https://fiber.skywire.dev) and is reliant upon on the output of certain cli commands ~~and some scripts~~
 
 [`skywire cli rewards ui`](cmd/skywire-cli/commands/rewards/ui.go) serves the reward system frontend or user interface - via http and dmsghttp.
 
-A wrapper script [`scripts/getlogs.sh`](scripts/getlogs.sh) is used to redirect the output of `skywire cli log` to a file, which is displayed at:
+The service which runs the reward system UI:
+/etc/systemd/system/[`fiberreward.service`](/scripts/rewards/services/fiberreward.service)
+
+A wrapper script [`getlogs.sh`](scripts/rewards/getlogs.sh) is used to redirect the output of `skywire cli log` to a file, which is displayed at:
 
 https://fiber.skywire.dev/log-collection
 
@@ -126,7 +105,7 @@ Here shows links to the reward calculations and distribution data by day:
 
 https://fiber.skywire.dev/skycoin-rewards
 
-on each linked page, the distribution data is displayed, and a link to the explorer for that transaction if it was broadcast. Also displayed are the public keys and their reward shares, or the reason why they were not rewarded
+on each linked page, the distribution data is displayed with a link to the explorer for that transaction if it was broadcast. Also displayed are the public keys and their reward shares, or the reason why they were not rewarded
 
 The frontend may be run either with flags or by using a conf file such as the following:
 fr.conf
@@ -166,5 +145,6 @@ REWARD_SYS_URL="dmsg://<reward-system-public-key>:80"
 
 before the script is run and the transaction is attempted to be broadcast, it's crucial to check that the hourly [log collection and reward calculation](https://fiber.skywire.dev/log-collection) is not ongoing.
 
+### Reward Notifications
 
-When the transaction is then broadcast, it's transaction ID recorded in a file which is monitored by the reward telegram bot, which generates a notification in https://t.me/skywire_reward. **Note: this will eventually be supplemented with or replaced by a notification via skychat.**
+When the transaction is broadcast by the reward system, it's transaction ID is recorded by appending a file which is monitored by the reward telegram bot. The telegram bot will then generate a notification in https://t.me/skywire_reward when a change to that file is detected. **Note: this will eventually be supplemented with or replaced by a notification via skychat.**

cmd/skywire-cli/commands/log/st.go

@@ -105,7 +105,7 @@ func makeTree() {
 			var coloredFile string
 			if kid == "proxy" {
 				testtime, _ := script.File(proxyCSV).Match(dirNode).Replace(",", " ").Column(2).String() //nolint
-				testres, _ := script.File(proxyCSV).Match(dirNode).Replace(",", " ").Column(3).String() //nolint
+				testres, _ := script.File(proxyCSV).Match(dirNode).Replace(",", " ").Column(3).String()  //nolint
 				if testtime != "" && testres != "" {
 					coloredFile = fmt.Sprintf("%s           %s	%s", pterm.Green("proxy"), strings.ReplaceAll(testtime, "\n", ""), strings.ReplaceAll(testres, "\n", ""))
 				} else {

cmd/skywire-cli/commands/rewards/tgbot.go

@@ -0,0 +1,117 @@
+// Package clirewards cmd/skywire-cli/commands/rewards/tgbot.go
+package clirewards
+
+import (
+	"fmt"
+	"log"
+	"os"
+	"strconv"
+	"strings"
+	"time"
+
+	"github.com/bitfield/script"
+	"github.com/spf13/cobra"
+	tele "gopkg.in/telebot.v3"
+)
+
+var filePath string
+
+func init() {
+	RootCmd.AddCommand(
+		tgbotCmd,
+	)
+	tgbotCmd.Flags().StringVarP(&filePath, "watch", "w", "../reward/rewards/transactions0.txt", "File to watch - file where reward transaction IDs are recorded")
+
+}
+
+var tgbotCmd = &cobra.Command{
+	Use:   "bot",
+	Short: "reward notification telegram bot",
+	Long:  "reward notification telegram bot",
+	Run: func(_ *cobra.Command, _ []string) {
+		chatIDStr := os.Getenv("TG_CHAT_ID")
+		chatID, err := strconv.ParseInt(chatIDStr, 10, 64)
+		if err != nil {
+			log.Fatalf("failed to parse chat ID: %v", err)
+		}
+		pref := tele.Settings{
+			Token:  os.Getenv("TG_BOT_TOKEN"),
+			Poller: &tele.LongPoller{Timeout: 10 * time.Second},
+		}
+
+		b, err := tele.NewBot(pref)
+		if err != nil {
+			log.Fatal(err)
+			return
+		}
+
+		tgbotscript := `#!/bin/bash
+		_stats() {
+		#  [[ $1 == "" ]] && return
+		  cat ` + strings.TrimSuffix(filePath, "/transactions0.txt") + `/hist/$(find ` + strings.TrimSuffix(filePath, "/transactions0.txt") + `/hist/ -name "*.txt" -type f -exec grep -l "$1" {} + | xargs -I{} basename {} | tr -d ".txt")_stats.txt
+		}
+		`
+
+		//	var lastModTime time.Time
+		lastModTime, err := os.Stat(filePath)
+		if err != nil {
+			log.Fatal(err)
+			return
+		}
+		// Use a goroutine to periodically check the file for changes
+		go func() {
+			for {
+				time.Sleep(2 * time.Second)
+
+				fileInfo, err := os.Stat(filePath)
+				if err != nil {
+					log.Printf("Error checking file info: %s", err)
+					continue
+				}
+
+				if fileInfo.ModTime().After(lastModTime.ModTime()) {
+					// The file has been modified since the last check, get the last line which is the most recent txid
+					lastLine, err := script.File(filePath).Last(1).String()
+					if err != nil {
+						log.Printf("Error getting last line of file: %v", err)
+						continue
+					}
+					if lastLine != "" {
+						tmpFile, err := os.CreateTemp(os.TempDir(), "*.sh")
+						if err != nil {
+							return
+						}
+						if err := tmpFile.Close(); err != nil {
+							return
+						}
+						_, _ = script.Exec(`chmod +x ` + tmpFile.Name()).String()                                                //nolint
+						_, _ = script.Echo(tgbotscript).WriteFile(tmpFile.Name())                                                //nolint
+						stats, err := script.Exec(`bash -c 'source ` + tmpFile.Name() + `  ; _stats ` + lastLine + `'`).String() //nolint
+						if err != nil {
+							log.Printf("Error getting statistics: %v", err)
+							continue
+						}
+						os.Remove(tmpFile.Name()) //nolint
+
+						dateforlink, err := script.Echo(stats).First(1).Replace("date: ", "").String()
+						if err != nil {
+							log.Printf("Error getting date for link: %v", err)
+							continue
+						}
+						msg := fmt.Sprintf("Rewards have been distributed!\n\nhttps://explorer.skycoin.com/app/transaction/%s\n\n%s\n\nhttps://fiber.skywire.dev/skycoin-rewards/hist/%s", lastLine, stats, dateforlink)
+						// Send the last line to the Telegram chat
+						_, err = b.Send(&tele.Chat{ID: chatID}, msg)
+						if err != nil {
+							log.Printf("Error sending message to Telegram chat: %s", err)
+							continue
+						}
+					}
+
+					lastModTime = fileInfo
+				}
+			}
+		}()
+
+		b.Start()
+	},
+}

cmd/skywire-cli/commands/rewards/ui.go

@@ -360,8 +360,8 @@ func server() {
 		tpstats, _ := script.Exec("skywire cli tp tree -s").Match("Count of transports:").Replace("Count of transports: ", "").Replace("\n", "").String() //nolint
 		tpcount, _ := strconv.Atoi(tpstats)                                                                                                               //nolint
 		if tpcount < 400 {
-			tpTree, _ := script.Exec("skywire cli tp tree").Bytes()  //nolint
-			c.Writer.Write(ansihtml.ConvertToHTML(tpTree)) //nolint
+			tpTree, _ := script.Exec("skywire cli tp tree").Bytes() //nolint
+			c.Writer.Write(ansihtml.ConvertToHTML(tpTree))          //nolint
 			c.Writer.Flush()
 		} else {
 			c.Writer.Write([]byte(fmt.Sprintf("Transport count: %v exceeds server resources to map", tpcount))) //nolint
@@ -851,7 +851,9 @@ func server() {
 		c.Writer.Header().Set("Transfer-Encoding", "chunked")
 		_, err := time.Parse("2006-01-02", c.Param("date"))
 		if err != nil {
-			if strings.Contains(c.Param("date"), "_rewardtxn0.csv") {
+			_, err1 := time.Parse("2006-01-02", strings.Replace(c.Param("date"), "_rewardtxn0.csv", "", -1))
+			_, err2 := time.Parse("2006-01-02", strings.Replace(c.Param("date"), "_stats.txt", "", -1))
+			if err1 != nil || err2 != nil {
 				filetoserve, err := script.File("rewards/hist/" + c.Param("date")).Bytes()
 				if err == nil {
 					c.Writer.Header().Set("Content-Type", "text/plain")
@@ -1261,7 +1263,7 @@ func serveSyntaxHighlighted(c *gin.Context) {
 		return
 	}
 	c.Status(http.StatusOK)
-	c.Writer.Write(buf.Bytes())  //nolint
+	c.Writer.Write(buf.Bytes()) //nolint
 }
 
 type ginHandler struct {

go.mod

@@ -55,6 +55,7 @@ require (
 	golang.org/x/sync v0.3.0
 	golang.org/x/sys v0.17.0
 	golang.zx2c4.com/wireguard v0.0.0-20230223181233-21636207a675
+	gopkg.in/telebot.v3 v3.2.1
 )
 
 require (