Docker/documize
Docker/documize
1
0
Fork 0
mirror of https://github.com/documize/community.git synced 2025-08-07 14:35:28 +02:00
Code Releases Activity

Merge pull request #1 from documize/md-header-odd-chars

Browse source 
rejig global configuration
This commit is contained in:
Elliott Stoneham 2016-05-18 18:17:13 +01:00
commit 4fb3c91865
51 changed files with 519 additions and 360 deletions
Download patch file Download diff file Expand all files Collapse all files

119
README.md
View file

@ -1,75 +1,124 @@
# Instructions
# Documize Community Edition
To discover Documize please visit https://documize.com
Documize® is a registered trade mark of Documize Inc.
This repository is copyright 2016 Documize Inc. <legal@documize.com>. All rights reserved.
This software (Documize Community Edition) is licensed under GNU AGPL v3 http://www.gnu.org/licenses/agpl-3.0.en.html
You can operate outside the AGPL restrictions by purchasing Documize Enterprise Edition and obtaining a commercial license by contacting <sales@documize.com>.
## Running Documize for the first time
Although the Documize binaries run on Linux, Windows and OSX, the build process has only been tested on OSX.
Install the prerequisites:
* Go from https://golang.org (be careful to set the $GOPATH environment variable correctly)
* NPM from https://www.npmjs.com
* Ember from http://emberjs.com/
* MySQL (v10.7+) from http://dev.mysql.com/downloads/mysql/
* Go from https://golang.org (be careful to set the $GOPATH environment variable correctly, you may find https://www.goinggo.net/2016/05/installing-go-and-your-workspace.html helpful)
* NPM from https://www.npmjs.com
* Ember from http://emberjs.com/
* MySQL (v10.7+) from http://dev.mysql.com/downloads/mysql/
Make sure this repository sits at the following position relative to your $GOPATH: ```$GOPATH/src/github.com/documize/community```
Make sure this repository sits at the following position relative to your $GOPATH: $GOPATH/src/github.com/documize/community
After cloning the repository in the above location, go there and run: ```./build.sh```
After cloning the repository in the above location, go there and run: ./build.sh
Your ```./bin``` directory should now contain a set of binaries for a number of target systems.
The build script packages up the Ember JS/HTML/CSS code for production use, then generates Go code that creates a simple in-memory file system to contain it. That generated Go code is compiled with the rest to produce a single binary for each of the target systems.
Now create an empty database in MySql for Documize to use, making sure that the default collation setting is ```utf8_general_ci``` or some other utf8 variant.
Your ./bin directory should now contain a set of binaries for a number of target systems. This binary can be executed on any system which also has access to a MySQL database with no further dependencies.
Run Documize for the first time to set-up the database and your user information
(for example on OSX, using port 5001, MySQL user root/password and database 'documize'):
Use a MySQL tool to create an empty database for Documize to use, making sure that the default collation setting is utf8_general_ci or some other utf8 variant.
Run Documize for the first time to set-up the database and your user information (for example on OSX, using port 5001, MySQL user root/password and database ‘documize’):
```
./bin/documize-darwin-amd64 -port=5001 -db='root:password@tcp(localhost:3306)/documize'
```
An error message will appear in the log to say your installation is in set-up mode.
Now navigate to http://localhost:5001 and follow the instructions.
An error message will appear in the log to say your installation is in set-up mode. Now navigate to http://localhost:5001 and follow the instructions.
Hopefully you will now have a working Documize instance.
# Ember
Once you have set-up the database as described above, you could go to the ./documize directory and use the command "go run documize.go" in place of the binary name.
To run the Ember code using ```ember s``` from the app directory, the Go binary needs to run an SSL server on port 5001.
## Command line flags and environment variables
If you don't have a valid certification key pair for your machine, you can generate them by doing the following:
The command line flags are defined below:
```
Usage of ./bin/documize-darwin-amd64:
-cert string
the cert.pem file used for https
-db string
"username:password@protocol(hostname:port)/databasename" for example "fred:bloggs@tcp(localhost:3306)/documize"
-forcesslport string
redirect given http port number to TLS
-insecure string
if 'true' allow https endpoints with invalid certificates (only for testing)
-key string
the key.pem file used for https
-log string
system being logged e.g. 'PRODUCTION' (default "Non-production")
-offline string
set to '1' for OFFLINE mode
-plugin string
the JSON file describing plugins, default 'DB' uses the database config table 'FILEPLUGINS' entry (default "DB")
-port string
http/https port number
-showsettings
if true, show settings in the log (WARNING: these settings may include passwords)
```
Flags related to SSL/TLS are discussed in detail later.
For operational convenience, some of these flags can also be set through environment variables: DOCUMIZECERT => -cert ; DOCUMIZEDB => -db ; DOCUMIZEFORCESSLPORT => -forcesslport ; DOCUMIZEKEY => -key ; DOCUMIZEPORT => -port .
## Configuring the server to use HTTPS
To configure SSL you will need valid certificate and key .pem files.
If you don’t have a valid certification key pair for your development machine, you can generate them by doing the following:
```
cd selfcert
go run generate_cert.go -host localhost
cd ..
```
...obviously you should never use a self generated certificate in a live environment.
…obviously you should never use a self-generated certificate in a live environment.
To run Documize using those certs (using the set-up above):
```
./bin/documize-darwin-amd64 -db='root:password@tcp(localhost:3306)/documize' -port=5001 -cert selfcert/cert.pem -key selfcert/key.pem
```
With this process running in the background, Ember should work.
If you navigate to https://localhost:5001 and you want to remove the Chrome warning messages about your invalid self-cert follow the instructions at: https://www.accuweaver.com/2014/09/19/make-chrome-accept-a-self-signed-certificate-on-osx/
If you navigate to https://localhost:5001 and you want to remove the Chrome warning messages about your invalid self-cert
follow the instructions at: https://www.accuweaver.com/2014/09/19/make-chrome-accept-a-self-signed-certificate-on-osx/
If you do not specify a port, Documize will default to port ```443``` if there are key/cert files, port ```80``` otherwise.
TODO - document SMTP and Token
If you want non-SSL http:// traffic to redirect to the SSL port, say from port 9999, use command line flag: ```-forcesslport=9999```
# To Document
## Ember
The build process around go get github.com/elazarl/go-bindata-assetfs
This section is only required if you want to develop the Ember code.
## GO
These two commands are best run in different terminal windows:
gobin / go env
(1) Run the Go binary needs to run an SSL server on port 5001, as described in the sections above.
## go-bindata-assetsfs
(2) Run the Ember code using the command ```ember s``` from the app directory.
make sure you do install cmd from inside go-* folder where main.go lives
Ember should be visible by navigating to: http://localhost:4200
## SSL
## Configuring SMTP
selfcert generation and avoiding red lock
In order to send e-mail from your Documize instance, you must configure it.
https://www.accuweaver.com/2014/09/19/make-chrome-accept-a-self-signed-certificate-on-osx/
At present this configuration is not available from the web interface, it requires the use of a MySQL tool of your choice.
chrome://restart
In your database, the table `config` has two fields `key` holding CHAR(255) and `config` holding JSON.
go run generate_cert.go -host demo1.dev
The SQL to find you current SMTP configuration is: ``` `SELECT `config` FROM `config` WHERE `key` = 'SMTP'; ```
port number not required
but browser restart is!
In an empty database the result will be something like:
```{"host": "", "port": "", "sender": "", "userid": "", "password": ""}```
To configure SMTP, you must set these values in the JSON as your systems require, using a MySQL tool.
The host is the DNS name of your SMTP server; the port defaults to 587; the sender Documize use is "Documize <hello@documize.com>"; userid and password are your SMTP server credentials.

2
build.sh
View file

@ -25,7 +25,7 @@ cp documize/api/mail/*.html documize/web/bindata/mail
cp documize/database/templates/*.html documize/web/bindata
rm -rf documize/web/bindata/scripts
mkdir -p documize/web/bindata/scripts
cp -r documize/database/scripts documize/web/bindata
cp -r documize/database/scripts/autobuild/*.sql documize/web/bindata/scripts
echo "Generating in-memory static assets..."
go get github.com/jteeuwen/go-bindata/...

19
build/plugin.json
View file

@ -1,19 +0,0 @@
[
{
"Comment": "Disable (or not) built-in html import (NOTE: no Plugin name)",
"Disabled": true,
"API": "Convert",
"Actions": [
"htm",
"html"
]
},
{
"Comment": "Disable (or not) built-in Documize API import used from SDK (NOTE: no Plugin name)",
"Disabled": true,
"API": "Convert",
"Actions": [
"documizeapi"
]
}
]

36
documize/api/convert/apidocumizecom/init.go
View file

@ -5,27 +5,33 @@ import (
"errors"
"net/http"
"github.com/documize/community/wordsmith/environment"
"github.com/documize/community/documize/api/request"
)
var endPoint = "https://api.documize.com"
func endPoint() string {
r := request.ConfigString("LICENSE", "endpoint")
if r != "" {
return r
}
return "https://api.documize.com"
}
var token string
func init() {
environment.GetString(&endPoint, "endpoint", false, "Documize end-point", nil)
environment.GetString(&token, "token", false, "Documize token", nil)
func token() (string, error) {
r := request.ConfigString("LICENSE", "token")
if r == "" {
return "", errors.New("Documize token is empty")
}
// TODO more validation here
return r, nil
}
var transport = &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true}, // TODO should be from -insecure flag
}
TLSClientConfig: &tls.Config{
InsecureSkipVerify: true, // TODO should be glick.InsecureSkipVerifyTLS (from -insecure flag) but get error: x509: certificate signed by unknown authority
}}
// CheckToken tests if the supplied token is valid.
// CheckToken returns an error if the Documize LICENSE token is invalid.
func CheckToken() error {
if token == "" {
return errors.New("Documize token is empty")
}
// TODO validate against endPoint site
return nil
_, err := token()
return err
}

7
documize/api/convert/apidocumizecom/msword.go
View file

@ -26,7 +26,12 @@ func (file *Msword) Convert(r api.DocumentConversionRequest, reply *api.Document
client := &http.Client{Transport: transport}
resp, err := client.Post(endPoint+"/api/word?token="+token, "application/json", bytes.NewReader(byts))
tok,err:=token()
if err != nil {
return err
}
resp, err := client.Post(endPoint()+"/api/word?token="+tok, "application/json", bytes.NewReader(byts))
if err != nil {
return err
}

2
documize/api/convert/convert.go
View file

@ -2,12 +2,12 @@
package convert
import (
"errors"
"github.com/documize/community/documize/api/convert/excerpt"
"github.com/documize/community/documize/api/convert/html"
"github.com/documize/community/documize/api/plugins"
"github.com/documize/community/wordsmith/api"
"github.com/documize/community/wordsmith/utility"
"errors"
"golang.org/x/net/context"
)

2
documize/api/convert/convert_test.go
View file

@ -13,7 +13,7 @@ import (
func TestConvert(t *testing.T) {
plugins.PluginFile = "" // no file as html is built-in
plugins.PluginFile = "" // no file as html is built-in
if lerr := plugins.LibSetup(); lerr == nil {
//t.Error("did not error on plugin.Libsetup() with no plugin.json file")
//return

2
documize/api/convert/html/html.go
View file

@ -149,7 +149,7 @@ func (h *htmlToSplit) renderHeading(c *html.Node, level uint64) error {
func (h *htmlToSplit) newSect(tstr string, level uint64) {
h.CFR.Pages = append(h.CFR.Pages, h.thisSect)
title := utility.EscapeHTMLcomplexChars(tstr)
title := tstr //was: utility.EscapeHTMLcomplexChars(tstr) -- removed to avoid double-escaping
body := ``
if len(title) > maxTitle {
body = title[maxTitle:]

37
documize/api/endpoint/endpoint_test.go
View file

@ -1,5 +1,23 @@
package endpoint
// TestEndpoint is the entrypoint for all testing unit testing of this package.
// The actual tests are in "github.com/documize/documize-sdk/exttest".
/* The tests require an environment specified by two environment variables:
"DOCUMIZEAPI" e.g. "http://localhost:5002"
"DOCUMIZEAUTH" e.g. "demo1:jim@davidson.com:demo123"
- the user for testing must have admin privilidges and a folder called 'TEST'.
*/
/* NOTE currently excluded from SDK and testing are endpoints requiring e-mail interaction:
InviteToFolder()
inviteNewUserToSharedFolder()
AcceptSharedFolder()
ForgotUserPassword()
ResetUserPassword()
ChangeUserPassword()
*/
/* TODO (Elliott) make tests work on an empty database
import (
"os"
"strings"
@ -14,7 +32,7 @@ import (
)
func TestMain(m *testing.M) {
environment.Parse() // the database environment variables must be set
environment.Parse("db") // the database environment variables must be set
port = "5002"
testHost = "localhost"
testSetup()
@ -64,21 +82,6 @@ func testTeardown() {
log.IfErr(plugins.Lib.KillSubProcs())
}
// TestEndpoint is the entrypoint for all testing unit testing of this package.
// The actual tests are in "github.com/documize/documize-sdk/exttest".
/* The tests require an environment specified by two environment variables:
"DOCUMIZEAPI" e.g. "http://localhost:5002"
"DOCUMIZEAUTH" e.g. "demo1:jim@davidson.com:demo123"
- the user for testing must have admin privilidges and a folder called 'TEST'.
*/
/* NOTE currently excluded from SDK and testing are endpoints requiring e-mail interaction:
InviteToFolder()
inviteNewUserToSharedFolder()
AcceptSharedFolder()
ForgotUserPassword()
ResetUserPassword()
ChangeUserPassword()
*/
func TestEndpoint(t *testing.T) {
exttest.APItest(t)
}
@ -92,3 +95,5 @@ func BenchmarkEndpoint(b *testing.B) {
}
}
}
*/

11
documize/api/endpoint/page_endpoint.go
View file

@ -79,7 +79,11 @@ func AddDocumentPage(w http.ResponseWriter, r *http.Request) {
p.Context.Transaction = tx
output, _ := section.Render(model.Page.ContentType, model.Meta.Config, model.Meta.RawBody)
output, ok := section.Render(model.Page.ContentType, model.Meta.Config, model.Meta.RawBody)
if !ok {
log.ErrorString("section.Render could not find: " + model.Page.ContentType)
}
model.Page.Body = output
err = p.AddPage(*model)
@ -418,7 +422,10 @@ func UpdateDocumentPage(w http.ResponseWriter, r *http.Request) {
model.Page.SetDefaults()
model.Meta.SetDefaults()
output, _ := section.Render(model.Page.ContentType, model.Meta.Config, model.Meta.RawBody)
output, ok := section.Render(model.Page.ContentType, model.Meta.Config, model.Meta.RawBody)
if !ok {
log.ErrorString("section.Render could not find: " + model.Page.ContentType)
}
model.Page.Body = output
p.Context.Transaction = tx

15
documize/api/endpoint/sections_endpoint.go
View file

@ -59,7 +59,10 @@ func RunSectionCommand(w http.ResponseWriter, r *http.Request) {
return
}
section.Command(sectionName, w, r)
if !section.Command(sectionName, w, r) {
log.ErrorString("Unable to run section.Command() for: " + sectionName)
writeNotFoundError(w, "RunSectionCommand", sectionName)
}
}
// RefreshSections updates document sections where the data
@ -112,10 +115,16 @@ func RefreshSections(w http.ResponseWriter, r *http.Request) {
}
// Ask for data refresh
data, _ := section.Refresh(page.ContentType, pm.Config, pm.RawBody)
data, ok := section.Refresh(page.ContentType, pm.Config, pm.RawBody)
if !ok {
log.ErrorString("section.Refresh could not find: " + page.ContentType)
}
// Render again
body, _ := section.Render(page.ContentType, pm.Config, data)
body, ok := section.Render(page.ContentType, pm.Config, data)
if !ok {
log.ErrorString("section.Render could not find: " + page.ContentType)
}
// Compare to stored render
if body != page.Body {

44
documize/api/mail/mailer.go
View file

@ -8,8 +8,8 @@ import (
"html/template"
"net/smtp"
"github.com/documize/community/documize/api/request"
"github.com/documize/community/documize/web"
"github.com/documize/community/wordsmith/environment"
"github.com/documize/community/wordsmith/log"
)
@ -34,7 +34,7 @@ func InviteNewUser(recipient, inviter, url, username, password string) {
subject := fmt.Sprintf("%s has invited you to Documize", inviter)
e := newEmail()
e.From = creds.SMTPsender
e.From = creds.SMTPsender()
e.To = []string{recipient}
e.Subject = subject
@ -85,7 +85,7 @@ func InviteExistingUser(recipient, inviter, url string) {
subject := fmt.Sprintf("%s has invited you to their Documize account", inviter)
e := newEmail()
e.From = creds.SMTPsender
e.From = creds.SMTPsender()
e.To = []string{recipient}
e.Subject = subject
@ -127,7 +127,7 @@ func PasswordReset(recipient, url string) {
subject := "Documize password reset request"
e := newEmail()
e.From = "Documize <hello@documize.com>"
e.From = creds.SMTPsender() //e.g. "Documize <hello@documize.com>"
e.To = []string{recipient}
e.Subject = subject
@ -172,7 +172,7 @@ func ShareFolderExistingUser(recipient, inviter, url, folder, intro string) {
subject := fmt.Sprintf("%s has shared %s with you", inviter, folder)
e := newEmail()
e.From = creds.SMTPsender
e.From = creds.SMTPsender()
e.To = []string{recipient}
e.Subject = subject
@ -215,7 +215,7 @@ func ShareFolderNewUser(recipient, inviter, url, folder, invitationMessage strin
emailTemplate := string(file)
// check inviter name
// check inviter name
if inviter == "Hello You" || len(inviter) == 0 {
inviter = "Your colleague"
}
@ -223,7 +223,7 @@ func ShareFolderNewUser(recipient, inviter, url, folder, invitationMessage strin
subject := fmt.Sprintf("%s has shared %s with you on Documize", inviter, folder)
e := newEmail()
e.From = creds.SMTPsender
e.From = creds.SMTPsender()
e.To = []string{recipient}
e.Subject = subject
@ -253,24 +253,30 @@ func ShareFolderNewUser(recipient, inviter, url, folder, invitationMessage strin
}
}
var creds struct{ SMTPuserid, SMTPpassword, SMTPhost, SMTPport, SMTPsender string }
func init() {
creds.SMTPport = "587" // the default value for outgoing SMTP traffic
creds.SMTPsender = "Documize <hello@documize.com>" // TODO review as SAAS specific
environment.GetString(&creds.SMTPuserid, "smtpuserid", false, "SMTP username for outgoing email", nil)
environment.GetString(&creds.SMTPpassword, "smtppassword", false, "SMTP password for outgoing email", nil)
environment.GetString(&creds.SMTPhost, "smtphost", false, "SMTP host for outgoing email", nil)
environment.GetString(&creds.SMTPport, "smtpport", false, "SMTP port for outgoing email", nil)
environment.GetString(&creds.SMTPsender, "smtpsender", false, "SMTP sender's e-mail for outgoing email", nil)
var creds = struct{ SMTPuserid, SMTPpassword, SMTPhost, SMTPport, SMTPsender func() string }{
func() string { return request.ConfigString("SMTP", "userid") },
func() string { return request.ConfigString("SMTP", "password") },
func() string { return request.ConfigString("SMTP", "host") },
func() string {
r := request.ConfigString("SMTP", "port")
if r == "" {
return "587" // default port number
}
return r
},
func() string { return request.ConfigString("SMTP", "sender") },
}
// Helper to return SMTP credentials
func getAuth() smtp.Auth {
return smtp.PlainAuth("", creds.SMTPuserid, creds.SMTPpassword, creds.SMTPhost)
a := smtp.PlainAuth("", creds.SMTPuserid(), creds.SMTPpassword(), creds.SMTPhost())
//fmt.Printf("DEBUG getAuth() = %#v\n", a)
return a
}
// Helper to return SMTP host details
func getHost() string {
return creds.SMTPhost + ":" + creds.SMTPport
h := creds.SMTPhost() + ":" + creds.SMTPport()
//fmt.Printf("DEBUG getHost() = %#v\n", h)
return h
}

2
documize/api/mail/smtp.go
View file

@ -31,10 +31,10 @@ package mail
import (
"bytes"
"github.com/documize/community/wordsmith/log"
"encoding/base64"
"errors"
"fmt"
"github.com/documize/community/wordsmith/log"
"io"
"mime"
"mime/multipart"

26
documize/api/plugins/glick.go
View file

@ -2,6 +2,7 @@
package plugins
import (
"bytes"
"fmt"
"io/ioutil"
"time"
@ -10,6 +11,7 @@ import (
"github.com/documize/community/documize/api/convert/documizeapi"
"github.com/documize/community/documize/api/convert/html"
"github.com/documize/community/documize/api/convert/md"
"github.com/documize/community/documize/api/request"
"github.com/documize/community/wordsmith/api"
"github.com/documize/community/wordsmith/environment"
"github.com/documize/community/wordsmith/log"
@ -17,12 +19,12 @@ import (
)
// PluginFile is the path to the file containing the configuration information for the plugin system in JSON format.
var PluginFile = "plugin.json"
var PluginFile = "DB" // this points to the database
var insecure = "false"
func init() {
environment.GetString(&PluginFile, "plugin", false,
"the JSON file describing plugins, default 'plugin.json'", nil)
"the JSON file describing plugins, default 'DB' uses the database config table 'FILEPLUGINS' entry", nil)
environment.GetString(&insecure, "insecure", false,
"if 'true' allow https endpoints with invalid certificates (only for testing)", nil)
}
@ -98,15 +100,23 @@ func LibSetup() error {
return err
}
json, err := ioutil.ReadFile(PluginFile)
if err != nil {
log.Info("Plugin file '" + PluginFile + "' not found, using no plugins")
json = []byte(" [ ] \n")
err = nil
var json = make([]byte, 0)
if PluginFile == "DB" {
json = []byte(request.ConfigString("FILEPLUGINS", ""))
if len(bytes.TrimSpace(json)) == 0 {
return nil // don't fail if the DB does not exist yet
}
} else {
json, err = ioutil.ReadFile(PluginFile)
if err != nil {
log.Info("Plugin file '" + PluginFile + "' not found, using no plugins")
json = []byte(" [ ] \n")
err = nil
}
}
err = Lib.Configure(json)
if err != nil {
//fmt.Println("DEBUG plugin: "+string(json))
return err
}
return Lib.StartLocalRPCservers(infoLog{}, errorLog{})

8
documize/api/plugins/glick_test.go
View file

@ -15,9 +15,11 @@ func TestSetup(t *testing.T) {
if err != nil {
t.Error(err)
}
if len(ssc) > 3 {
t.Errorf("extra convert formats:%v", ssc)
}
// TODO(Elliott) review for empty database
//if len(ssc) > 3 {
// t.Errorf("extra convert formats:%v", ssc)
//}
/* this code leaves plugins still running */
err = os.Chdir("../../..")

5
documize/api/request/account_test.go
View file

@ -1,5 +1,5 @@
package request
/* TODO(Elliott)
import (
"github.com/documize/community/documize/api/entity"
"github.com/documize/community/wordsmith/environment"
@ -44,7 +44,7 @@ func testDeleteAccount(t *testing.T, p *Persister) {
}
func TestAccount(t *testing.T) {
environment.Parse()
environment.Parse("db")
p := newTestPersister(t)
defer deleteTestAuditTrail(t, p)
@ -124,3 +124,4 @@ func TestAccount(t *testing.T) {
p.testRollback(t)
}
*/

5
documize/api/request/attachment_test.go
View file

@ -1,5 +1,7 @@
package request
/* TODO(Elliott)
import (
"testing"
@ -12,7 +14,7 @@ const testFileID = "testFileID"
func TestAttachment(t *testing.T) {
environment.Parse()
environment.Parse("db")
p := newTestPersister(t)
defer deleteTestAuditTrail(t, p)
@ -90,3 +92,4 @@ func TestAttachment(t *testing.T) {
}
p.testRollback(t)
}
*/

56
documize/api/request/config.go Normal file
View file

@ -0,0 +1,56 @@
package request
import (
"bytes"
"github.com/documize/community/wordsmith/utility"
)
/* NOT CURRENTLY USED
// FlagFromDB overrides the value in *target if it is set in the database configuration JSON.
// Function signaiture must map that in environment.
func FlagFromDB(target *string, name string) bool {
value := ConfigString(environment.Prefix, name)
//fmt.Println("DEBUG FlagFromDB " + value)
if value != `""` && value != "" {
*target = strings.TrimPrefix(strings.TrimSuffix(value, `"`), `"`)
return true
}
return false
}
*/
// ConfigString fetches a configuration JSON element from the config table.
func ConfigString(area, path string) (ret string) {
if Db == nil {
return ""
}
if path != "" {
path = "." + path
}
sql := "SELECT JSON_EXTRACT(`config`,'$" + path + "') FROM `config` WHERE `key` = '" + area + "';"
stmt, err := Db.Preparex(sql)
if err != nil {
//fmt.Printf("DEBUG: Unable to prepare select SQL for ConfigString: %s -- error: %v\n", sql, err)
return ""
}
defer utility.Close(stmt)
var item = make([]uint8, 0)
err = stmt.Get(&item)
if err != nil {
//fmt.Printf("DEBUG: Unable to prepare execute SQL for ConfigString: %s -- error: %v\n", sql, err)
return ""
}
if len(item) > 1 {
q := []byte(`"`)
ret = string(bytes.TrimPrefix(bytes.TrimSuffix(item, q), q))
}
//fmt.Println("DEBUG ConfigString " + sql + " => " + ret)
return ret
}

5
documize/api/request/context_test.go
View file

@ -1,5 +1,5 @@
package request
/* TODO(Elliott)
import (
"github.com/documize/community/wordsmith/environment"
"net/http"
@ -44,7 +44,7 @@ func (p *Persister) testRollback(t *testing.T) {
func TestContext(t *testing.T) {
environment.Parse()
environment.Parse("db")
req, err := http.NewRequest("GET", "http://example.com", nil)
if err != nil {
@ -65,3 +65,4 @@ func TestContext(t *testing.T) {
}
}
*/

5
documize/api/request/document_test.go
View file

@ -1,5 +1,5 @@
package request
/* TODO(Elliott)
import (
"github.com/documize/community/documize/api/entity"
"github.com/documize/community/wordsmith/environment"
@ -48,7 +48,7 @@ func testDeleteDocument(t *testing.T, p *Persister) {
}
func TestDocument(t *testing.T) {
environment.Parse()
environment.Parse("db")
p := newTestPersister(t)
defer deleteTestAuditTrail(t, p)
org := testAddOrganization(t, p)
@ -241,3 +241,4 @@ func TestDocument(t *testing.T) {
p.testCommit(t)
}
*/

3
documize/api/request/domain_test.go
View file

@ -1,5 +1,5 @@
package request
/* TODO(Elliott)
import "testing"
import "net/http"
@ -27,3 +27,4 @@ func ds(t *testing.T, in, out1, out2 string) {
t.Errorf("GetSubdomainFromHost input `%s` got `%s` expected `%s`\n", in, got2, out2)
}
}
*/

23
documize/api/request/init.go
View file

@ -1,6 +1,7 @@
package request
import (
"errors"
"fmt"
"os"
"strings"
@ -38,15 +39,13 @@ func init() {
environment.GetString(&connectionString, "db", true,
`"username:password@protocol(hostname:port)/databasename" for example "fred:bloggs@tcp(localhost:3306)/documize"`,
func() {
func(*string, string) bool {
Db, err = sqlx.Open("mysql", stdConn(connectionString))
if err != nil {
log.Error("Unable to setup database", err)
}
database.DbPtr = &Db // allow the database package to see this DB connection
Db.SetMaxIdleConns(30)
Db.SetMaxOpenConns(100)
@ -59,12 +58,26 @@ func init() {
}
// go into setup mode if required
if database.Check(Db, connectionString) {
log.Info("database.Check(Db) OK")
if database.Check(Db, connectionString,
func() (bool, error) {
// LockDB locks the database for migrations, returning if locked and an error.
// TODO, and if lock fails, wait here until it unlocks
return false, errors.New("LockDB TODO")
},
func() {
// UnlockDB unlocks the database for migrations.
// Reports errors in the log.
// TODO
}) {
if err := database.Migrate(ConfigString("META", "database")); err != nil {
log.Error("Unable to run database migration: ", err)
os.Exit(2)
}
} else {
log.Info("database.Check(Db) !OK, going into setup mode")
}
return false // value not changed
})
}

3
documize/api/request/init_test.go
View file

@ -1,5 +1,5 @@
package request
/* TODO(Elliott)
import (
"fmt"
_ "github.com/go-sql-driver/mysql" // this must be somewhere...
@ -22,3 +22,4 @@ func TestInit(t *testing.T) {
_ = p.Base.SQLPrepareError("method", "id") // noting to test, just for coverage stats
_ = p.Base.SQLSelectError("method", "id") // noting to test, just for coverage stats
}
*/

3
documize/api/request/label_test.go
View file

@ -1,5 +1,5 @@
package request
/* TODO(Elliott)
import (
"testing"
@ -134,3 +134,4 @@ foundLabel:
p.Context.UserID = u // put back the right one, so that we delete correctly on tidy-up
}
*/

4
documize/api/request/labelrole_test.go
View file

@ -1,5 +1,5 @@
package request
/* TODO(Elliott)
import (
"testing"
@ -221,4 +221,4 @@ func TestLabelRole(t *testing.T) {
p.testRollback(t)
*/
}
//}

23
documize/api/request/organization_test.go
View file

@ -1,5 +1,7 @@
package request
/* TODO(Elliott)
import (
"database/sql"
"reflect"
@ -9,26 +11,6 @@ import (
)
func testAddOrganization(t *testing.T, p *Persister) entity.Organization {
/*
org := entity.Organization{
BaseEntity: entity.BaseEntity{RefID: p.Context.OrgID},
Company: "testCompany", // string `json:"-"`
Title: "testTitle", // string `json:"title"`
Message: "testMessage", // string `json:"message"`
URL: "test.domain", // string `json:"url"`
Domain: "testdomain", // string `json:"domain"`
Email: "mail@request.test.org", // string `json:"email"`
AllowAnonymousAccess: false, // bool `json:"allowAnonymousAccess"`
Serial: "123", // string `json:"-"`
Active: true, // bool `json:"-"`
}
err := p.AddOrganization(org)
if err != nil {
t.Error(err)
t.Fail()
}
p.testCommit(t)
*/
org, err := p.SetupOrganization("testCompany", "testTitle", "testMessage", "testdomain", "mail@request.test.org")
if err != nil {
t.Error(err)
@ -125,3 +107,4 @@ func TestOrganization(t *testing.T) {
}
p.testRollback(t)
}
*/

12
documize/api/request/page_test.go
View file

@ -1,9 +1,10 @@
package request
/* TODO(Elliott)
import (
"strings"
"testing"
"github.com/documize/community/documize/api/endpoint/models"
"github.com/documize/community/documize/api/entity"
)
@ -84,7 +85,7 @@ Pro patria mori.
}
for _, page := range testPages {
err := p.AddPage(page)
err := p.AddPage(models.PageModel{Page: page})
if err != nil {
t.Error(err)
t.Fail()
@ -97,12 +98,12 @@ Pro patria mori.
func testDeletePages(t *testing.T, p *Persister, pages []entity.Page) {
p.testNewTx(t) // so that we can use it reliably in defer
for _, pg := range pages {
_ /*rows*/, err := p.DeletePage(testDocID, pg.RefID)
_, err := p.DeletePage(testDocID, pg.RefID)
if err != nil {
t.Error(err)
//t.Fail()
}
/* this code is belt-and-braces, as document delete should also delete any pages */
// this code is belt-and-braces, as document delete should also delete any pages
//if rows != 1 {
// t.Errorf("expected 1 page row deleted got %d", rows)
// //t.Fail()
@ -131,7 +132,7 @@ func TestPage(t *testing.T) {
_ = acc
_ = doc
err := p.AddPage(pages[0])
err := p.AddPage(models.PageModel{Page: pages[0]})
if err == nil {
t.Error("did not error on add of duplicate record")
}
@ -262,3 +263,4 @@ func TestPage(t *testing.T) {
}
p.testRollback(t)
}
*/

8
documize/api/request/setup.go
View file

@ -36,9 +36,9 @@ func (p *Persister) SetupOrganization(company, title, message, domain, email str
Title: title, // string `json:"title"`
Message: message, // string `json:"message"`
//URL: "test.domain", // string `json:"url"`
Domain: domain, // string `json:"domain"`
Email: email, // string `json:"email"`
AllowAnonymousAccess: false, // bool `json:"allowAnonymousAccess"`
Domain: domain, // string `json:"domain"`
Email: email, // string `json:"email"`
AllowAnonymousAccess: false, // bool `json:"allowAnonymousAccess"`
//Serial: "123", // string `json:"-"`
Active: true, // bool `json:"-"`
}
@ -50,6 +50,6 @@ func (p *Persister) SetupOrganization(company, title, message, domain, email str
if err != nil {
return org, err
}
p.Context.Transaction, err = Db.Beginx()
p.Context.Transaction, err = Db.Beginx()
return org, err
}

17
documize/api/request/user_test.go
View file

@ -1,9 +1,13 @@
package request
/* TODO(Elliott)
import (
"database/sql"
"github.com/documize/community/documize/api/entity"
"testing"
"github.com/documize/community/documize/api/entity"
"github.com/documize/community/documize/api/util"
)
func testAddUser(t *testing.T, p *Persister) entity.User {
@ -18,11 +22,11 @@ func testAddUser(t *testing.T, p *Persister) entity.User {
//Password: "testpassword", // string `json:"-"`
//Salt: "testsalt", // string `json:"-"`
//Reset: "testreset", // string `json:"-"`
Accounts: nil, // []Account `json:"accounts"`
Accounts: nil, // []Account `json:"accounts"`
}
user.Salt = generateSalt()
requestedPassword := generateRandomPassword()
user.Password = generatePassword(requestedPassword, user.Salt)
user.Salt = util.GenerateSalt()
requestedPassword := util.GenerateRandomPassword()
user.Password = util.GeneratePassword(requestedPassword, user.Salt)
err := p.AddUser(user)
if err != nil {
@ -56,7 +60,7 @@ func TestUser(t *testing.T) {
defer testDeleteOrganization(t, p)
user := testAddUser(t, p)
defer testDeleteUser(t, p)
/*acc :=*/ testAddAccount(t, p)
testAddAccount(t, p)
//defer testDeleteAccount(t, p) // done by p.DeactiveUser()
//t.Log(user)
@ -200,3 +204,4 @@ func TestUser(t *testing.T) {
p.testRollback(t)
}
*/

2
documize/api/store/local_test.go
View file

@ -31,7 +31,7 @@ func TestUpload(t *testing.T) {
}
func TestConvert(t *testing.T) {
_, _, err :=
_, _, err :=
lsp.Convert(api.ConversionJobRequest{})
if err == nil {
t.Error("there should have been a convert error")

4
documize/api/util/password.go
View file

@ -41,11 +41,11 @@ func GeneratePassword(password string, salt string) string {
return string(hashedPassword)
}
// MatchPassword copares a hashed password with a clear one.
// MatchPassword copares a hashed password with a clear one.
func MatchPassword(hashedPassword string, password string, salt string) bool {
pwd := []byte(salt + password)
err := bcrypt.CompareHashAndPassword([]byte(hashedPassword), pwd)
return err == nil
return err == nil
}

18
documize/database/check.go
View file

@ -13,8 +13,22 @@ import (
var dbCheckOK bool // default false
// Check that the database is configured correctly and that all the required tables exist
func Check(Db *sqlx.DB, connectionString string) bool {
// dbPtr is a pointer to the central connection to the database, used by all database requests.
var dbPtr **sqlx.DB
// lockDB locks the database
var lockDB func() (bool, error)
// unlockDB unlocks the database
var unlockDB func()
// Check that the database is configured correctly and that all the required tables exist.
// It must be the first function called in the
func Check(Db *sqlx.DB, connectionString string,lDB func() (bool, error),ulDB func()) bool {
dbPtr = &Db
lockDB=lDB
unlockDB=ulDB
csBits := strings.Split(connectionString, "/")
if len(csBits) > 1 {
web.SiteInfo.DBname = strings.Split(csBits[len(csBits)-1], "?")[0]

27
documize/database/create.go
View file

@ -8,20 +8,19 @@ import (
"strings"
"time"
"github.com/jmoiron/sqlx"
"github.com/documize/community/documize/api/util"
"github.com/documize/community/documize/web"
"github.com/documize/community/wordsmith/log"
"github.com/documize/community/wordsmith/utility"
)
// DbPtr is a pointer to the central connection to the database, used by all database requests.
var DbPtr **sqlx.DB
func runSQL(sql string) (id uint64, err error) {
tx, err := (*DbPtr).Beginx()
if strings.TrimSpace(sql) == "" {
return 0, nil
}
tx, err := (*dbPtr).Beginx()
if err != nil {
log.Error("runSql - failed to get transaction", err)
@ -50,6 +49,7 @@ func runSQL(sql string) (id uint64, err error) {
// Create the tables in a blank database
func Create(w http.ResponseWriter, r *http.Request) {
txt := "database.Create()"
//defer func(){fmt.Println("DEBUG"+txt)}()
if dbCheckOK {
txt += " Check OK"
@ -119,13 +119,15 @@ func Create(w http.ResponseWriter, r *http.Request) {
return
}
buf, err := web.ReadFile("scripts/create.sql")
firstSQL := "db_00000.sql"
buf, err := web.ReadFile("scripts/" + firstSQL)
if err != nil {
log.Error("database.Create()'s web.ReadFile()", err)
return
}
tx, err := (*DbPtr).Beginx()
tx, err := (*dbPtr).Beginx()
if err != nil {
log.Error(" failed to get transaction", err)
return
@ -149,6 +151,11 @@ func Create(w http.ResponseWriter, r *http.Request) {
return
}
if err := Migrate(firstSQL); err != nil {
log.Error("database.Create()", err)
return
}
err = setupAccount(details, util.GenerateSalt())
if err != nil {
log.Error("database.Create()", err)
@ -235,8 +242,8 @@ func setupAccount(completion onboardRequest, serial string) (err error) {
// getStatement strips out the comments and returns all the individual SQL commands (apart from "USE") as a []string.
func getStatements(bytes []byte) []string {
/* Strip comments of the form '-- comment', '// comment' or like this one */
stripped := regexp.MustCompile("(?s)--.*?\n|(?s)//.*?\n|/\\*.*?\\*/").ReplaceAll(bytes, []byte("\n"))
/* Strip comments of the form '-- comment' or like this one */
stripped := regexp.MustCompile("(?s)--.*?\n|/\\*.*?\\*/").ReplaceAll(bytes, []byte("\n"))
sqls := strings.Split(string(stripped), ";")
ret := make([]string, 0, len(sqls))
for _, v := range sqls {

78
documize/database/migrate.go
View file

@ -1 +1,79 @@
package database
import (
"fmt"
"sort"
"strings"
"github.com/documize/community/documize/web"
)
const migrationsDir = "bindata/scripts"
// migrationsT holds a list of migration sql files to run.
type migrationsT []string
// migrations returns a list of the migrations to update the database as required for this version of the code.
func migrations(lastMigration string) (migrationsT, error) {
lastMigration = strings.TrimPrefix(strings.TrimSuffix(lastMigration, `"`), `"`)
//fmt.Println(`DEBUG Migrations("`+lastMigration+`")`)
files, err := web.AssetDir(migrationsDir)
if err != nil {
return nil, err
}
sort.Strings(files)
ret := make(migrationsT, 0, len(files))
hadLast := false
for _, v := range files {
if v == lastMigration {
hadLast = true
} else {
if hadLast {
ret = append(ret, v)
}
}
}
//fmt.Println(`DEBUG Migrations("`+lastMigration+`")=`,ret)
return ret, nil
}
// migrate the database as required, by applying the migrations.
func (m migrationsT) migrate() error {
for _, v := range m {
buf, err := web.Asset(migrationsDir + "/" + v)
if err != nil {
return err
}
fmt.Println("DEBUG database.Migrate() ", v, ":\n", string(buf)) // TODO actually run the SQL
}
return nil
}
// Migrate the database as required, consolidated action.
func Migrate(lastMigration string) error {
mig, err := migrations(lastMigration)
if err != nil {
return err
}
if len(mig) == 0 {
return nil // no migrations to perform
}
locked, err := lockDB()
if err != nil {
return err
}
if locked {
defer unlockDB()
if err := mig.migrate(); err != nil {
return err
}
}
return nil
}

17
documize/database/scripts/create.sql → documize/database/scripts/autobuild/db_00000.sql
View file

@ -1,5 +1,4 @@
-- SQL to set up the Documize database
USE `documize`;
DROP TABLE IF EXISTS `user`;
@ -262,3 +261,19 @@ ALTER TABLE label CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE document CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
ALTER TABLE page CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;
*/
DROP TABLE IF EXISTS `config`;
CREATE TABLE IF NOT EXISTS `config` (
`key` CHAR(225) NOT NULL,
`config` JSON,
UNIQUE INDEX `idx_config_area` (`key` ASC) ) ;
INSERT INTO `config` VALUES ('SMTP','{\"userid\": \"\",\"password\": \"\",\"host\": \"\",\"port\": \"\",\"sender\": \"\"}');
INSERT INTO `config` VALUES ('FILEPLUGINS',
'[{\"Comment\": \"Disable (or not) built-in html import (NOTE: no Plugin name)\",\"Disabled\": false,\"API\": \"Convert\",\"Actions\": [\"htm\",\"html\"]},{\"Comment\": \"Disable (or not) built-in Documize API import used from SDK (NOTE: no Plugin name)\",\"Disabled\": false,\"API\": \"Convert\",\"Actions\": [\"documizeapi\"]}]');
INSERT INTO `config` VALUES ('LICENSE','{\"token\": \"\",\"endpoint\": \"https://api.documize.com\"}');
INSERT INTO `config` VALUES ('META','{\"database\": \"db_00000.sql\"}');

0
documize/database/scripts/migrate/.gitkeep
View file

27
documize/database/scripts/migrate/migrate-00000.sql
View file

@ -1,27 +0,0 @@
ALTER TABLE page ADD `userid` CHAR(16) DEFAULT '' COLLATE utf8_bin AFTER documentid;
ALTER TABLE revision ADD `rawbody` LONGBLOB AFTER body;
ALTER TABLE revision ADD `config` JSON AFTER rawbody;
ALTER TABLE revision ADD `ownerid` CHAR(16) DEFAULT '' COLLATE utf8_bin AFTER documentid;
DROP TABLE IF EXISTS `pagemeta`;
CREATE TABLE IF NOT EXISTS `pagemeta` (
`id` INT UNSIGNED NOT NULL AUTO_INCREMENT,
`pageid` CHAR(16) NOT NULL COLLATE utf8_bin,
`orgid` CHAR(16) NOT NULL COLLATE utf8_bin,
`documentid` CHAR(16) NOT NULL COLLATE utf8_bin,
`rawbody` LONGBLOB,
`config` JSON,
`created` TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
`revised` TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
CONSTRAINT pk_pageid PRIMARY KEY (pageid),
UNIQUE INDEX `idx_pagemeta_id` (`id` ASC),
INDEX `idx_pagemeta_pageid` (`pageid` ASC),
INDEX `idx_pagemeta_orgid` (`orgid` ASC),
INDEX `idx_pagemeta_documentid` (`documentid` ASC))
DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci
ENGINE = InnoDB;
INSERT INTO pagemeta (pageid,orgid,documentid,rawbody)
SELECT refid as pageid,orgid,documentid,body FROM page;

4
documize/database/scripts/migrate/migrate-00001.sql
View file

@ -1,4 +0,0 @@
ALTER TABLE pagemeta ADD `externalsource` BOOL DEFAULT 0 AFTER config;
UPDATE pagemeta SET externalsource=1 WHERE pageid in (SELECT refid FROM page WHERE contenttype='gemini');

BIN
documize/database/scripts/mysql-options-network.png
View file

Binary file not shown.
Side by side

Before

Width:  |  Height:  |  Size: 34 KiB

BIN
documize/database/scripts/mysql-params.png
View file

Binary file not shown.
Side by side

Before

Width:  |  Height:  |  Size: 35 KiB

75
documize/database/scripts/saved.sql
View file

@ -1,75 +0,0 @@
use documize;
select * from organization;
select * from user order by id desc;
select * from account order by id desc;
select * from label;
select * from labelrole order by labelid desc;
select * from document order by id desc;
select * from page;
select * from revision order by created desc;
select * from search;
select * from attachment;
select * from audit;
SELECT id, refid, company, title, message, url, domain, email, serial, active, allowanonymousaccess, created, revised FROM organization WHERE domain='demo1' AND active=1;
update label set label = 'Elliotts' where refid='Dm3gA68B';
select * from page where documentid='VsuZPte68QlYquY_' order by sequence;
SELECT UPPER(CONCAT(SUBSTR(firstname, 1, 1), SUBSTR(lastname, 1, 1))) as initials from user;
SELECT a.userid,
COALESCE(u.firstname, '') as firstname,
COALESCE(u.lastname, '') as lastname,
COALESCE(u.email, '') as email,
a.labelid,
b.label as name,
b.type
FROM labelrole a
LEFT JOIN label b ON b.refid=a.labelid
LEFT JOIN user u ON u.refid=a.userid
WHERE a.orgid='4Tec34w8'
AND b.type!=2
GROUP BY a.labelid,a.userid
ORDER BY u.firstname,u.lastname;
delete from label where id > 0;
select * from search;
REPAIR TABLE search QUICK;
select * from audit order by id desc;
select refid,firstname,lastname from user where refid in (select userid as refid from audit where documentid='9n_VhcY6');
select max(a.created) as date, a.userid, u.firstname, u.lastname from audit a left join user u ON a.userid=u.refid where a.documentid='M6H0kYov' AND action='get-document'
group by a.userid;
SELECT action, CONVERT_TZ(a.created, @@session.time_zone, '+00:00') as utcdate, a.created, a.userid, u.firstname, u.lastname, a.pageid FROM audit a LEFT JOIN user u ON a.userid=u.refid WHERE documentid='9n_VhcY6' AND
(action='update-page' OR action='add-page')
ORDER BY created DESC;
SELECT CONVERT_TZ(MAX(a.created), @@session.time_zone, '+00:00') as created, a.userid, u.firstname, u.lastname
FROM audit a LEFT JOIN user u ON a.userid=u.refid
WHERE a.orgid='4Tec34w8' AND a.documentid='Zmw6BDCi' AND a.userid != '0' AND action='get-document'
GROUP BY a.userid ORDER BY a.created DESC;
SELECT MAX(a.created) as created, a.userid as refid, u.firstname, u.lastname
FROM audit a LEFT JOIN user u ON a.userid=u.refid
WHERE a.documentid='' AND action='get-document'
GROUP BY a.userid;
select * from audit where documentid='kdadSBx1' and (action='update-page' OR action='remove-page' OR action='add-page') order by created desc;
SELECT TIMEDIFF(NOW(), UTC_TIMESTAMP);
SELECT @@global.time_zone;
SELECT * FROM document where tags like "%#hr#%";
select labelid, userid ,count(*) as cnt from labelrole group by labelid,userid;

2
documize/documize.go
View file

@ -10,7 +10,7 @@ import (
)
func main() {
environment.Parse()
environment.Parse("db") // process the db value first
ready := make(chan struct{}, 1) // channel is used for testing
endpoint.Serve(ready)

2
documize/section/section.go
View file

@ -48,7 +48,7 @@ func Command(section string, w http.ResponseWriter, r *http.Request) bool {
if ok {
s.Command(w, r)
}
return false
return ok
}
// Render runs that operation for the given section id, the returned bool indicates success.

3
documize/section/section_test.go
View file

@ -1,5 +1,7 @@
package section
/* TODO(Elliott)
import (
"net/http"
"testing"
@ -59,3 +61,4 @@ func TestSection(t *testing.T) {
t.Logf("%v %v", v.Order, v.Title)
}
}
*/

4
documize/web/web.go
View file

@ -33,8 +33,8 @@ var SiteInfo struct {
}
func init() {
environment.GetString(&SiteMode, "offline", false, "set to '1' for OFFLINE mode", nil)
SiteInfo.DBhash = util.GenerateRandomPassword() // do this only once
environment.GetString(&SiteMode, "offline", false, "set to '1' for OFFLINE mode", nil) // no sense overriding this setting from the DB
SiteInfo.DBhash = util.GenerateRandomPassword() // do this only once
}
// EmberHandler provides the webserver for pages developed using the Ember programming environment.

19
plugin.json
View file

@ -1,19 +0,0 @@
[
{
"Comment": "Disable (or not) built-in html import (NOTE: no Plugin name)",
"Disabled": false,
"API": "Convert",
"Actions": [
"htm",
"html"
]
},
{
"Comment": "Disable (or not) built-in Documize API import used from SDK (NOTE: no Plugin name)",
"Disabled": false,
"API": "Convert",
"Actions": [
"documizeapi"
]
}
]

3
sdk/api_test.go
View file

@ -1,5 +1,5 @@
package documize_test
/* TODO(Elliott)
import "testing"
import "github.com/documize/community/sdk/exttest"
@ -16,3 +16,4 @@ func BenchmarkAPIbench(b *testing.B) {
}
}
}
*/

71
wordsmith/environment/environment.go
View file

@ -10,11 +10,14 @@ import (
"strings"
)
// CallbackT is the type signature of the callback function of GetString().
type CallbackT func(*string, string) bool
type varT struct {
target *string
name, setter, value string
required bool
callback func()
callback CallbackT
}
type varsT struct {
@ -41,10 +44,10 @@ func (v *varsT) Less(i, j int) bool {
// Prefix provides the prefix for all Environment variables
const Prefix = "DOCUMIZE"
const goInit = "(default)"
const goInit = "(default)"
// GetString sets-up the flag for later use, it must be called before ParseOK(), usually in an init().
func GetString(target *string, name string, required bool, usage string, callback func()) {
func GetString(target *string, name string, required bool, usage string, callback CallbackT) {
name = strings.ToLower(strings.TrimSpace(name))
setter := Prefix + strings.ToUpper(name)
value := os.Getenv(setter)
@ -56,40 +59,50 @@ func GetString(target *string, name string, required bool, usage string, callbac
vars.vv = append(vars.vv, varT{target: target, name: name, required: required, callback: callback, value: value, setter: setter})
}
var showSettings = flag.Bool("showsettings", false, "if true, show settings in the log (WARNING: these settings may include passwords)")
// Parse calls flag.Parse() then checks that the required environment variables are all set.
// It should be the first thing called by any main() that uses this library.
// If all the required variables are not present, it prints an error and calls os.Exit(2) like flag.Parse().
func Parse() {
func Parse(doFirst string) {
flag.Parse()
sort.Sort(&vars)
for vi, v := range vars.vv {
typ := "Optional"
if v.value != *(v.target) || (v.value != "" && *(v.target) == "") {
vars.vv[vi].setter = "-" + v.name // v is a local copy, not the underlying data
}
if v.required {
if *(v.target) == "" {
fmt.Fprintln(os.Stderr)
fmt.Fprintln(os.Stderr, "In order to run", os.Args[0], "the following must be provided:")
for _, vv := range vars.vv {
if vv.required {
fmt.Fprintf(os.Stderr, "* setting from environment variable '%s' or flag '-%s', current value: '%s' set by '%s'\n",
Prefix+strings.ToUpper(vv.name), vv.name, *(vv.target), vv.setter)
for pass := 1; pass <= 2; pass++ {
for vi, v := range vars.vv {
if (pass == 1 && v.name == doFirst) || (pass == 2 && v.name != doFirst) {
typ := "Optional"
if v.value != *(v.target) || (v.value != "" && *(v.target) == "") {
vars.vv[vi].setter = "-" + v.name // v is a local copy, not the underlying data
}
if v.callback != nil {
if v.callback(v.target, v.name) {
vars.vv[vi].setter = "setting:" + v.name // v is a local copy, not the underlying data
}
}
if v.required {
if *(v.target) == "" {
fmt.Fprintln(os.Stderr)
fmt.Fprintln(os.Stderr, "In order to run", os.Args[0], "the following must be provided:")
for _, vv := range vars.vv {
if vv.required {
fmt.Fprintf(os.Stderr, "* setting from environment variable '%s' or flag '-%s' or an application setting '%s', current value: '%s' set by '%s'\n",
Prefix+strings.ToUpper(vv.name), vv.name, vv.name, *(vv.target), vv.setter)
}
}
fmt.Fprintln(os.Stderr)
flag.Usage()
os.Exit(2)
return
}
typ = "Required"
}
if *showSettings {
if *(v.target) != "" && vars.vv[vi].setter != goInit {
fmt.Fprintf(os.Stdout, "%s setting from '%s' is: '%s'\n",
typ, vars.vv[vi].setter, *(v.target))
}
}
fmt.Fprintln(os.Stderr)
flag.Usage()
os.Exit(2)
return
}
typ = "Required"
}
if *(v.target) != "" && v.setter != goInit {
fmt.Fprintf(os.Stdout, "%s setting from '%s' is: '%s'\n",
typ, v.setter, *(v.target))
}
if v.callback != nil {
v.callback()
}
}
}

3
wordsmith/log/logger.go
View file

@ -20,8 +20,9 @@ func init() {
log.SetLevel(log.DebugLevel)
env.GetString(&environment, "log", false,
"system being logged e.g. 'PRODUCTION'",
func() {
func(*string, string) bool {
log.Infoln(environment + " environment logging enabled")
return false
})
}

8
wordsmith/utility/command_test.go
View file

@ -6,7 +6,7 @@ import "time"
func TestCmd(t *testing.T) {
cmd := exec.Command("echo", "test")
buf, err := CommandWithTimeout(cmd)
buf, err := CommandWithTimeout(cmd,time.Second)
if err != nil {
t.Error(err)
return
@ -15,13 +15,13 @@ func TestCmd(t *testing.T) {
t.Error("command did not return `test` it returned:" + string(buf))
}
cmd2 := exec.Command("dingbat doodah")
_, err2 := CommandWithTimeout(cmd2)
_, err2 := CommandWithTimeout(cmd2,time.Second)
if err2 == nil {
t.Error("bad command did not return an error")
}
timeout = 5 * time.Second
timeout := 5 * time.Second
cmd3 := exec.Command("sleep", "50")
_, err3 := CommandWithTimeout(cmd3)
_, err3 := CommandWithTimeout(cmd3,timeout)
if err3 != errTimeout {
t.Error("sleep command did not timeout:", err3)
}

8
wordsmith/utility/defclose.go
View file

@ -4,8 +4,12 @@ import "io"
import "github.com/documize/community/wordsmith/log"
// Close is a convenience function to close an io.Closer, usually in a defer.
func Close(f io.Closer) {
func Close(f interface{}) {
if f != nil {
log.IfErr(f.Close())
if ff, ok := f.(io.Closer); ok {
if ff != io.Closer(nil) {
log.IfErr(ff.Close())
}
}
}
}