Introduction

This is a home project, a should not be used in a production environment type project.

The scope of what I'm trying to do here is a simple one, its to bring in the chat services I use into a single client. A similar concept we had back in the day with Pidgin.IM.

I've been working on how to do this for a while, and spent a lot of time using the Mattermost project, however once I'd got the Slack Bridge working, the other bridges were very flaky, and the experience has not been great on my setup.

Objective

The end result of this project has been to create my own riot.im server with matrix-synapse authentication and utilise the Matrix bridges available to bring in WhatsApp, Slack, Telegram and Discord at very least into a single chat application I can use to communicate with others from

Roadmap

Step 1 - Create a self hosted Riot front end for Matrix - Done
Step 2 - Create a self hosted Matrix server - Done
Step 3 - Add Bridges to consolidate my communications nightmare into a single interface - Working On
Step 4 - Migrate this setup to use Postgres - Done

Setup

I chose to set this up using docker where possible on top of OpenSUSE Leap 15.1 server (not that that should make a difference) so I can test quickly the various components and work through issues quickly without the need to rebuild the server each time.

I do not profess to be a docker expert, I get things working, and have provided the docker run commands below, if someone wants to help get this into a single docker compose file, amazing.

The write up here will also not cover port forward from the outside world to the inside world on my router. If you use the instructions the assumption is you know how to setup port forwarding on your home router.

Note: I'm also opening up ports on the box i run docker on as it's separate from the box I run Nginx on, the Nginx sits on a different subnet in reality to the docker box.

Assumptions

  • Riot URL riot.myserver.com
  • Matrix/Synapse URL - matrix.myserver.com
  • Docker installed and running
  • Certbot installed and running
  • Understanding of how port forwarding works on your router
  • SSH Access to the server with root access

What is Riot.IM?

Riot.im is a Free and open-source software instant messaging client based on the Matrix protocol and is distributed under the Apache License 2.0. Because it uses the federated Matrix protocol, Riot.im lets the user choose a server to connect to.
Additionally, Riot.im supports end to end encryption, groups, channels and sharing of files between users. It is available as a web application, as desktop apps for all major operating systems and as a mobile app for Android and iOS. The development of the app is primarily done by the company New Vector Limited, which is also involved in the development of the Matrix protocol itself

Installing Riot.IM homeserver

The first task to get get the self hosted riot.im server running in my homelab this will be the server I access from the Internet.

Installing Riot Server

The following command sets up riot-web on port 80 using /etc/riot-web/config.json locally

Note: I've used port 80 here, in a production system, even behind a reverse proxy, this is not a good idea.
docker run -p 80:80 -v /etc/riot-web/config.json:/app/config.json vectorim/riot-web

Config File

The file /etc/riot-web/config.json has the following content

{
"default_server_name": "matrix.org",
"brand": "Riot",
"integrations_ui_url": "https://scalar-staging.vector.im/",
"integrations_rest_url": "https://scalar-staging.vector.im/api",
"integrations_widgets_urls": [
    "https://scalar.vector.im/_matrix/integrations/v1",
    "https://scalar.vector.im/api",
    "https://scalar-staging.vector.im/_matrix/integrations/v1",
    "https://scalar-staging.vector.im/api",
    "https://scalar-staging.riot.im/scalar/api"
],
"hosting_signup_link": 
"https://modular.im/?utm_source=riot-web&utm_medium=web",
"bug_report_endpoint_url": "https://riot.im/bugreports/submit",
"features": {
    "feature_pinning": "labs",
    "feature_custom_status": "labs",
    "feature_custom_tags": "labs",
    "feature_state_counters": "labs",
    "feature_many_integration_managers": "labs",
    "feature_mjolnir": "labs",
    "feature_dm_verification": "labs",
    "feature_cross_signing": "enable",
    "feature_invite_only_padlocks": "enable",
    "feature_event_indexing": "disable",
    "feature_bridge_state": "labs",
    "feature_presence_in_room_list": "labs",
    "feature_custom_themes": "labs"
},
"piwik": {
    "url": "https://piwik.riot.im/",
    "siteId": 1,
    "policyUrl": "https://matrix.org/legal/riot-im-cookie-policy"
},
"roomDirectory": {
    "servers": [
        "matrix.org"
    ]
},
"enable_presence_by_hs_url": {
    "https://matrix.org": false,
    "https://matrix-client.matrix.org": false
},
"terms_and_conditions_links": [
    {
        "url": "https://riot.im/privacy",
        "text": "Privacy Policy"
    },
    {
        "url": "https://matrix.org/legal/riot-im-cookie-policy",
        "text": "Cookie Policy"
    }
]
}

This is the stock config file

Firewall

The ports need opening on opensuse to allow the reverse proxy access.

firewall-cmd --add-port=80/tcp --permanent
firewall-cmd --reload

Reverse Proxy

From the internet we would like this server accessible from https and to do this, I've put an Nginx reverse proxy and use a LetsEncrypt certificate

Setup the conf file

nano /etc/nginx/sites-available/riotserver.conf

add the following

server {
server_name riot.myserver.com;
set $upstream 192.168.20.11:80;
location / {
proxy_pass_header Authorization;
proxy_pass http://$upstream;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_http_version 1.1;
proxy_set_header Connection “”;
proxy_buffering off;
client_max_body_size 0;
proxy_read_timeout 36000s;
proxy_redirect off;
}
}

Make sure your port forwarding is setup and test from the internet.

If your port forwarding is successful then we can run certbot and register the URL with an SSL Cert

certbot --nginx -d riot.myserver.com

Certbot will run through some tests on port 80, confirm it can see what it needs to see and then provide you with the option to redirect all http traffic to https (Option2) select option 2 and check https://riot.myserver.com

You should now be able to login using a matrix account to your own Riot server.

If all you want it to host your own web front end to Matrix, you could stop here.

This setup was derived from the following information

vector-im/riot-web
A glossy Matrix collaboration client for the web. Contribute to vector-im/riot-web development by creating an account on GitHub.

Installing Postgres Docker

By default the Matrix Synapse server installs with SQLite and its advised to only use this for testing, the setup

Setup Container

Pull the Docker Image

docker pull postgres

Run the Docker Container

docker run --name matrix-postgres -e POSTGRES_PASSWORD=EnTeRaPaSsWoRd -d postgres

I noted that the docker logs for the Postgres image stated

initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.

I'd suggest on a production system you'd need to lock this down

Connect to Docker Postgres Server

You will need to jump onto the Postgres server and create a user and a database on the server

docker ps

This will show the containers and ID's for each container you'll need the ID for the container

dfed2da49afc        postgres                       "docker-entrypoint.s…"   38 minutes ago      Up 38 minutes       5432/tcp  

Then connect to the docker container's bash console

docker exec -it dfed2da49afc bash

This will connect to the containers bash shell

Switch to the Postgres user

su - postgres

Setup Postgres User

We need to create a database owner for the Synapse Database we will create

createuser --pwprompt synapse_user

You will be prompted for a password, enter something you can remember we will need this later:

Enter password for new role:

Setup Postgres DB

Now the Postgres owner is setup, the database needs to be created, this is done using the psql tool

psql

now run the command to create the database and assign the synapse_user you created earlier

CREATE DATABASE synapse
 ENCODING 'UTF8'
 LC_COLLATE='C'
 LC_CTYPE='C'
 template=template0
 OWNER synapse_user;

Check User and Database

We can check the user created using the command

\du

To check the database has been created and has synapse_user as the owner run

\l

The Database is created

You will need to note the IP of the Docker container, the database name, user and database password

docker instpect dfed2da49afc

Look for the line which states:

"IPAddress": "172.17.0.4",
matrix-org/synapse
Synapse: Matrix reference homeserver. Contribute to matrix-org/synapse development by creating an account on GitHub.

Installing Matrix Synapse Server

While the riot server will happily connect to the Matrix servers for new accounts and authentication and hosting of rooms. For most of the Bridges I wanted to run, having my own 'homeserver' Matrix Synapse build seemed to be the way to go.

While not a pig to get working once you know how, if you don't the instructions around this product leave a little to be desired and are written with a lot of assumed knowlege.

Again like the riot server the easiest way of getting Synapse working was docker

Generate Configs

Before we run the server container, we need to generate some configs, the docker run command uses the local folder mapped to /home/user/synapsedata/ which Synapse will create config files in

sudo docker run -it --rm --mount type=volume,src=synapse-data,dst=/data -e SYNAPSE_SERVER_NAME=mymatrix.safewebbox.com -e SYNAPSE_REPORT_STATS=yes matrixdotorg/synapse:latest generate

This will create the sqlitedb and config files for the matrix server under

sudo cd /home/user/synapsedata/

the contents of this directory should look something like this

-rw-r--r-- 1  991  991 147005440 Apr 19 20:05 homeserver.db
-rw-r--r-- 1  991  991     57968 Apr 19 20:05 homeserver.db-journal
-rw-r--r-- 1  991  991     63655 Apr 19 20:01 homeserver.yaml
drwxr-xr-x 1  991  991        60 Apr 19 19:52 media_store
-rw-r--r-- 1 root root       573 Apr 19 16:39 mymatrix.safewebbox.com.log.config
-rw-r--r-- 1  991  991        59 Apr 19 16:39 mymatrix.safewebbox.com.signing.key

Edit the homeserver.yaml

Out of the box there are some changes needed, I've not copied the whole homeserver.yaml the lines below however are the ones I changed in the file. I would suggest taking a copy of the homeserver.yaml before changing it.

Lines relating to servers

server_name: "matrix.myserver.com"

public_baseurl: https://matrix.myserver.com/


client_base_url: "https://riot.myserver.com"

Need to Enable Registration or you can't create a new account on the server

enable_registration: true

Allowing specific email adresses

allowed_local_pids:
medium: emailpattern: '.*@matrix.org'
medium: emailpattern: '.*@myemaildomain.co.uk'

Trust relationships with other servers

trusted_third_party_id_servers:
    matrix.org
    vector.im
trusted_key_servers:
    server_name: "matrix.org"

Setup Email Server (Using internal mail relay)

email:
    smtp_host: 192.168.20.9
    smtp_port: 25
    notif_from: "Your Friendly %(app)s homeserver david@myemaildomain.com"
app_name: my_mydomain_matrix_server

Finally we need to change the database from SQLite to Postgres

database:
name: psycopg2
args:
    user: synapse_user
    password: ENTER THE PASSWORD YOU USED
    database: synapse
    host: 172.17.0.4
    cp_min: 5
    cp_max: 10

The password is the one you entered when you setup the synapse_user account earlier and the host is the docker IP of the postgres container you noted earlier.

Save and exit this file

Run docker

Now the homeserver.yaml is configured we can run the matrix docker server

sudo docker run -d --name synapse --mount type=volume,src=synapse-data,dst=/data -p 8008:8008 matrixdotorg/synapse:latest

you can check the logs using

sudo docker logs synapse

Open firewall

Open a firewall port from the OS to allow traffic through to docker

sudo firewall-cmd --add-port=8008/tcp --permanent
sudo firewall --reload

Setup reverse proxy

Like the Riot server above create a basic conf file for nginx

sudo nano /etc/nginx/sites-available/matrixserver.conf

Add

server {
server_name matrix.mydomain.com;
set $upstream 192.168.86.11:8008;
location / {
proxy_pass_header Authorization;
proxy_pass http://$upstream;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_http_version 1.1;
proxy_set_header Connection “”;
proxy_buffering off;
client_max_body_size 0;
proxy_read_timeout 36000s;
proxy_redirect off;
}
}

restart nginx

sudo systemctl restart nginx

Make sure your port forwarding is setup and test from the internet.

If your port forwarding is successful then we can run certbot and register the URL with an SSL Cert

certbot --nginx -d matrix.myserver.com

Certbot will run through some tests on port 80, confirm it can see what it needs to see and then provide you with the option to redirect all http traffic to https (Option2) select option 2 and check https://matrix.myserver.com

Setup SRV record

To federate with other servers, Matrix relies on having a SVR record setup for your DNS

On hover.com this setup looks like this.

Once the SVR record is setup

The service also needs to listen on port 8448 for the federation to work properly as well, so you'll need to setup your router to talk to your nginx reverse proxy on port 8448/tcp

We then setup a /etc/nginx/sites-available/matrix8448.conf

sudo nano /etc/nginx/sites-available/matrix8448.conf

Use the following config which listens on port 8448 but redirects to port 8008

Note: You can do this as an additional server block in the matrix.conf we created earlier, and in production that would be the better way to do it, i have split this out to help me do some troubleshooting and i have a personal thing about per port nginx configs. I find it easier. Just me.
server_name matrix.mydomain.com;
set $upstream 192.168.20.11:8008;
 location / {
 proxy_pass_header Authorization;
 proxy_pass http://$upstream;
 proxy_set_header Host $host;
 proxy_set_header X-Real-IP $remote_addr;
 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
 proxy_http_version 1.1;
 proxy_set_header Connection “”;
 proxy_buffering off;
 client_max_body_size 0;
 proxy_read_timeout 36000s;
 proxy_redirect off;
}

listen 8448 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/matrix.mydomain.com/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/matrix.mydomain.com/privkey.pem; 
include /etc/letsencrypt/options-ssl-nginx.conf; 
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; 

}
server {
if ($host = matrix.mydomain.com) {
    return 301 https://$host$request_uri;
}

}

I've hijacked the SSL cert i generated previously for the matrix.conf so I don't need to run certbot again.

Test federation

The federation between your new Matrix-synapse server and public matrix servers can be tested by going to

https://federationtester.matrix.org/

enter the domain name you want to test (no https)

matrix.mydomain.com

If working you should get

Got 1 connection report.
Homeserver version: Synapse 1.12.3


Connection Reports
188.198.81.126:8448   ChecksSuccess

No SRV Records
View the json report.

At this point we have a working Riot.IM and Matrix Synapse Server

Logging in

Note:

As was suggested by /u/driminicus on reddit while the instructions below take you through changing the default server from matrix.org to matrix.mydomain.com you can make this easier by editing the riot-web json file
nano  /etc/riot-web/config.json
and change
"default_server_name": "matrix.org",
to
"default_server_name": "matrix.mydomain.com",
Which will default the Matrix server you've just setup, restart the docker container for the changes to take effect.

Connect to your riot server

https://riot.mydomain.com

Click on Change next to matrix.org

Enter the URL of the Synapse server https://matrix.mydomain.com

Click on Create account

Register a new account, setup the encryption and you're done and should now be able to login to riot using the new account against your matrix server.

Observations from the Internet

I've had some amazing feedback by some people on Reddit and other internet places i've posted this which may help others, i'm not going to update all the instructions, however this is feedback which may help you if you're going to try this.

Using domain.tld instead of subdomain.domain.tld

From:  /u/driminicus on reddit

You set the homeserver name to matrix.myserver.com, which (as you noticed) works fine, but it is similar to having user@email.myserver.com, it works but it does look a bit silly to have the needless email subdomain.
To fix this you can either use .well-known delegation, or SRV delegation. I generally recommend using .well-known delegation, as that is usually easier. This means you'll want to host a json file on https://myserver.com/.well-known/matrix/server with the content { "m.server": "matrix.myserver.com:443"} (or you can of course point to a different port).
If you use nginx you can even do something like: location /.well-known/matrix/server { return 200 '{"m.server": "matrix.myserver.com:443"}' }
If you want to use an SRV record for domain delegation you have to make sure your server is listening with a cert valid for myserver.com (and not just matrix.myserver.com), because DNS is unreliable.

Using Postgres Instead of SQLite

From:  /u/driminicus on reddit

Please don't use sqlite for a federated homeserver! you'll really want to use postgresql, sqlite result in fairly bad performance, increasing the time it takes to send and receive messages. sqlite support is really only advised for local test servers.

My intention was not to use this as community server, however this observation is correct and i'll be working out how to get Postgres into this setup over the next few weeks

matrix-org/synapse
Synapse: Matrix reference homeserver. Contribute to matrix-org/synapse development by creating an account on GitHub.

https://federationtester.matrix.org/