# TJ CSL This is the documentation site for the TJHSST Computer Systems Lab. If you are contributing for the first time, please read our [documentation guide](/general/documentation). For an overview of the Sysadmin program, see the [Sysadmin page](/general). This is intended for TJHSST Student System Administrators only. # Services # Ion The TJ Intranet platform ## Ion Ion () is the next-generation Intranet system used at TJHSST. Using Python, Django, Redis, Postgres, and many other technologies, Ion was developed from the ground up to be simple, well-documented, and extensible. Ion allows students, teachers, and staff at TJHSST to access student information, manage activity signups, and view information on news and events. [Read more about how Ion is used at Thomas Jefferson](https://ion.tjhsst.edu/about). {% hint style="info" %} Note: Ion is not to be referred to as "ION" in capital letters. The correct spelling is "Ion". {% endhint %} ## History Ion was completely student-built and was the senior research project of James Woglom (Class of 2016). It was first announced on Wednesday, November 11th, 2015 concurrently with a [tribute to Iodine developers](https://web.archive.org/web/20151111231602/https://iodine.tjhsst.edu/) on the Iodine homepage. [Iodine was](https://twitter.com/TJIntranet/status/664857149324005377) [shut down](https://twitter.com/TJIntranet/status/665273342396604417) the evening of Friday November 13th, and both Iodine and Ion remained inaccessible throughout the weekend. After completing teacher training, Ion was [officially released](https://twitter.com/TJIntranet/status/666356448801251330) at the end of the school day on Monday November 16th, one day before initially planned. It ran its first Eighth Period block successfully [two days later](https://twitter.com/TJIntranet/status/666619609143975936) on Wednesday November 18th. ## Architecture Ion is a Django application backed by a PostgreSQL database and using Redis to perform in-memory caching. ## Contributing The contact person for Intranet is the [Intranet Lead](/general/sysadmins-list#current-leads). # Development {% hint style="info" %} Knowledge of Git and basic Linux commands is a prerequisite for Ion development. {% endhint %} Before continuing, you should probably read about Ion's architecture. Before continuing, it would also be useful to know Python, Django, and the client-server model (or at least have documentation, references, and tutorials available). ## Resources {% embed url="" %} {% embed url="" %} {% embed url="" %} {% embed url="" %} # Overview This section contains guidance for creating the Ion Development Environment in your own laptop or PC. Please also check out [Ion's setup instructions](https://github.com/tjcsl/ion/blob/master/SETUP.md) for additional guidance. If in doubt, follow the instructions in the official Ion repository. # Setup You have two options when it comes to developing for Ion: {% content-ref url="/pages/-LKatwtRXVU2Pew51f3Z" %} [Vagrant Setup](/services/ion/development/setup/vagrant-setup) {% endcontent-ref %} {% content-ref url="/pages/rF1RsnyeCe5gydE9MeFn" %} [Docker Setup](/services/ion/development/setup/docker-setup) {% endcontent-ref %} # Docker Setup ## Setting up Docker Docker is one of the two options you have for managing Ion's development environment. Docker is a software platform that is used to build applications using containers. If you want to learn more about Docker, refer to its [documentation](https://docs.docker.com/). First, download and install Git from [here](https://git-scm.com/downloads) if you are on Windows. Ensure you have an SSH key set up with GitHub by running `ssh -T git@github.com`. You should be greeted by your username. If not, set up an SSH key with GitHub by following [these instructions](https://help.github.com/articles/generating-an-ssh-key/). Install Docker by following [these instructions](https://www.docker.com/products/docker-desktop/) based on your operating system. For Windows and Mac users, install Docker Desktop. After installing Git and Docker, fork the [`tjcsl/ion`](https://github.com/tjcsl/ion) repository, and clone your forked Ion repository*.* Once the cloning completed, `cd` into the `intranet` directory. Note that a more up to date guide will be located at . ``` $ git clone git@github.com:/ion.git intranet $ cd intranet ``` {% hint style="info" %} Note: if your host machine is running Windows, please run `git config core.autocrlf input` before cloning to prevent line ending issues. {% endhint %} Also make sure that Docker Compose is installed along with `docker`, this is important when it comes to building and bringing the development environment to life. Make sure that during this process you are in the config/docker directory. Run `docker compose build` to get all the dependences that Ion needs, like `redis`, `postgres`, `celery`, etc. This should take a minute or two. Once after the building is complete, run `docker compose up` to compose the container. You can add `-d` flag to the command if you wish. ### Post Setup Navigate to [http://localhost:8080](http://localhost:8080/) in the web browser of your choice. If this is the first time composing the container (in which in this case; running `docker-compose up -d` for the first time on that computer), you may have to wait for approximately 60 seconds for it to start running. When presented with the login page, you can login in the default admin account which is `admin`/`notfish`. ### Useful Commands #### Interacting with the application: If you need to run a Django command like `makemigrations`, `collectstatic` or `shell_plus`, run `docker exec -it intranet bash` in your terminal. That will give you a shell into the application container. You can also use this to run scripts like `build_sources.sh`. If you need to view the output from or restart `runserver`, run `docker attach application`. ### History The use of Docker for developing Ion was anticipated for a long time after Ion was first release in 2015. The former Vagrant environment didn't really had a multitude of problems, but the idea of making a VM and configuring the VM was a bit annoying for many Sysadmins and understudies who wanted to build the development environment. The Development of the new environment started in early 2022 when then-understudy Justin Lee (`2025jlee`) developed way to make a Ion Development Environment using Docker. The Docker environment solved some of the problems that Vagrant had, most notably the speed building the environment. Around early 2023, the Docker environment went through a second round of updating to make the environment more quicker and easier to build. # Vagrant Setup ## Setting up Vagrant Vagrant is used to manage Ion's development environment so that it closely resembles the production environment. To get started, download and install git from [here](https://git-scm.com/downloads) if you are running Windows or git is not installed. After that, download and install Virtualbox from [here](https://www.virtualbox.org/wiki/Downloads) and Vagrant from [here](http://docs.vagrantup.com/v2/installation/index.html). When you're installing Vagrant, you should install it on your host OS, running nested VMs is not recommended. Ensure you have an SSH key set up with GitHub by running `ssh -T git@github.com`. You should be greeted by your username. If not, set up an SSH key with GitHub by following [these instructions](https://help.github.com/articles/generating-an-ssh-key/). With Vagrant and Virtualbox installed, clone the Ion repository onto the host computer and `cd` into the new directory. Note that a more up to date guide will be located at . {% hint style="info" %} Note: if your host machine is running Windows, please run `git config core.autocrlf input` before cloning to prevent line ending issues. {% endhint %} ``` $ git clone git@github.com:tjcsl/ion.git intranet $ cd intranet ``` In the `config/vagrant` directory, copy the file `devconfig.json.sample` to `devconfig.json` and edit the properties in `devconfig.json` as appropriate. Ensure `ssh_key` is set to the same SSH key registered with GitHub (e.g. `id_rsa`). Also make sure that `use_nfs` is set to `true` and `use_vpn` is set to `false`. This will prevent connecting to the CSL VPN. {% hint style="info" %} The other values specified in `devconfig.json` are optional. The `ldap_simple_bind_password` is not needed and is a remnant of the old LDAP-based authentication scheme. {% endhint %} {% hint style="info" %} Connecting to the CSL VPN may be necessary to test Kerberos authentication or other functionality that requires connection to CSL services. In that case, `use_vpn` should be set to `true`. {% endhint %} Run `vagrant plugin install vagrant-vbguest vagrant-bindfs` If you are on Windows, also run `vagrant plugin install vagrant-winnfsd`. Run `vagrant up && vagrant reload` and wait while the development environment is set up. When asked to select a network interface for bridging, enter the number corresponding to one that is active. To automatically select this interface in the future, set the "network\_interface" key in `devconfig.json` to the name of the interface you selected (e.g. `"en0: Wi-Fi (AirPort)"`). There may be repeated warnings similar to "`Remote connection disconnect` on the second `vagrant up`. After several minutes they will stop. Once the provisioning process is complete, run `vagrant ssh` to log in to the development box. Move into the `intranet` directory and run `workon ion` to load the Python dependencies. `workon ion` should always be the first thing you run after you SSH into the development box. The Git repository on the host computer is synced with `~/intranet` on the virtual machine, so you can edit files within the repo on the host computer with a text editor of your choice and the changes will be immediately reflected on the virtual machine. ### Troubleshooting If you get a `SIOCADDRT: Network is unreachable` error when running `vagrant up`, you need to start the OpenVPN client. If you see a `Adding routes to host computer...` message, you probably forgot to start the OpenVPN client. ## Post-Setup After successfully setting up the Vagrant environment, you will want to actually access your sandbox. Start by connecting to the Vagrant box using `vagrant ssh`. (Consider running all of the following in a `screen` or `tmux` session.) Make sure you’re in the `intranet` directory, and run `python manage.py migrate`. This will set up the Postgres database. You can then start the built-in Django web server with `fab runserver`. Now that you are running the development server, open a browser to and log in. If it fails, check the output of `manage.py runserver`. ### Setting Up Groups Currently, there are no default groups set up when you first install Ion. In order to grant yourself administrative privileges, you must be a member of the `admin_all` group. To create and add yourself to the global administrator group, run the following commands: ```python $ ./manage.py shell_plus Python 3.5.2 (default, Nov 23 2017, 16:37:01) Type 'copyright', 'credits' or 'license' for more information IPython 6.2.1 -- An enhanced Interactive Python. Type '?' for help. >>> user = User.objects.get_or_create(username="YOURUSERNAME")[0] >>> group = Group.objects.get_or_create(name="admin_all")[0] >>> user.groups.add(group) >>> user.is_superuser = True >>> user.save() ``` ### Connecting and Disconnecting from the VM When you want to close the VM environment, make sure you have exited out of the ssh session and then run `vagrant suspend`. To resume the session, run `vagrant resume`. Suspending and resuming is significantly faster than halting and starting, and also dumps the contents of the machine’s RAM to disk. ### Setting up Files You can find a list of file systems at `intranet/apps/files/models.py`. To add these systems so that they appear on the Files page, run the statements found in the file. A sample is shown below: ```python $ ./manage.py shell_plus Python 3.5.2 (default, Nov 23 2017, 16:37:01) Type 'copyright', 'credits' or 'license' for more information IPython 6.2.1 -- An enhanced Interactive Python. Type '?' for help. >>> Host.objects.create(name="Computer Systems Lab", code="csl", address="remote.tjhsst.edu", linux=True) ``` ### Increasing RAM With any RAM lower than the default 2048MB, you may run into performance constraints. If you encounter signifigant issues, it is recommended to bump the VM’s amount of memory, through VirtualBox Manager, to at least that amount. ## Reasons for Vagrant Setup The Ion developers decided to use Vagrant to manage Ion's development environment because it closely modeled Ion's production server. Vagrant also enables the quick creation of VMs with various customizations possible. # Environment Now that you have set up your development environment, you should get familiar with your environment and the Ion code base. ## Useful Commands ### Vagrant To manage your Vagrant box, you should use the following commands: | Command | Description | | ------- | ----------- | | `vagrant suspend` | Saves the state of the VM | | ----------------- | ------------------------- | | `vagrant resume` | Resumes the previous state of theVM | | ---------------- | ----------------------------------- | | `vagrant ssh` | SSHs into the VM | | ------------- | ---------------- | | `vagrant reload` | Halts the VM and then brings it back up | | ---------------- | --------------------------------------- | | `vagrant up` | Brings up the VM according to specified `Vagrantfile` | | ------------ | ----------------------------------------------------- | | `vagrant destroy` |

(A DANGEROUS COMMAND)

Stops the VM & permanently destroys the VM and its contents

| | ----------------- | ---------------------------------------------------------------------------------------------- | To manage the Vagrant box within the box, you should use the following commands: | Command | Description | | --------------------------- | ------------------------------------- | | `workon ion` | Initialize virtualenvwrapper | | `fab runserver` | Run a server in development | | `./manage.py migrate` | Migrate the database if not already | | `./manage.py shell_plus` | Enter a Python shell | | `./manage.py collectstatic` | Collect static files into a directory | You should ALWAYS make sure `workon ion` has been run after SSH-ing into the VM. The database should be migrated fairly regularly (especially after changes in the database/models) in development. Running `fab runserver` should make the server accessible from `localhost:8080` (on your host machine's web browser). Running `./setup.py test` should test your current code base against the Ion test suite. ## Code base ## Overall Structure ### Root The root of the Ion git repository is split into many parts in order to allow developers to access information quickly: * `config`: Contains scripts/files used provision the Vagrant box * `cron`: Contains cron bash scripts that are only run on production * `docs`: Contains old Ion docs * `intranet`: Contains most of Ion's code * `Ion.egg-info`: Contains information about the Python Ion eggs * `migrations`: Skeleton directory only created for migrations * `scripts`: Contains scripts useful for Ion developers Some useful files in the root of the Ion git repository include: * `COPYING`: Contains the GPLv2+ for the Ion code * `fabfile.py`: Describes behavior of `fab`. Used for development/deployment. * `manage.py`: Contains wrapper for Django shell management commands * `README.rst`: Contains the README * `requirements.txt`: Contains Ion's dependencies * `setup.py`: Describes the Python Ion package * `Vagrantfile`: Describes configuration of the Vagrant box ## Intranet Some useful sub directories of the `intranet` directory include (as per Django best practices): * `apps`: Contains the vast majority of Django apps * `middleware`: Contains Django middleware * `settings`: Contains Django settings for Ion * `static`: Contains CSS, images, JS, SVGs, themes, and useful documents * `templates`: Contains Django templates and email templates * `test`: Contains the Ion base test suite * `utils`: Contains side-wide utilities All other files in this directory are per Django convention. If you do not know what they do, Google it. Within the `apps` directory, there are multiple Django apps with descriptive names. # Fixtures Fixtures are SQL queries that use a real production data set to populate the development database. Since this contains personal information, access should be restricted. Fixtures may be obtained (through a secure method) from an Ion admin if you are developing for Ion. ## Import Once you get a zipped fixtures file, unzip the file to a `fixtures` directory in your main work directory. Within the main work directory, run `./scripts/import_fixtures.sh` to import the fixtures. This process may take some time. ## Export To export fixtures from production, you should SSH to root on `ion`. Navigate to the main directory, make a directory called `fixtures`, and run `./scripts/export_fixtures.sh`. Your fixtures should be populated in the `fixtures` directory and can now be zipped for transfer. # PR Workflow ## Prerequisites Knowledge of Git is good to have for Ion development given that we store code in Git and use GitHub. Note that the most up to date version of this guide will be at . ## Branching Production code is stored on the `master` branch in the [Ion repo on GitHub](https://github.com/tjcsl/ion). The `dev` branch stores code used for testing before deploying to production. Non-Ion maintainers should develop on their own forks of the main GitHub repository. You can read about it [here](https://help.github.com/articles/fork-a-repo/). ## Preparation When you think your code is ready to reviewed, it is important to ensure your changes comply with the Ion code guidelines. Here are some general guidelines * First, you should review your code for compliance with the [Ion Style Guide](/services/ion/development/style-guide). * Second, you should make sure that only changes you wanted to make exist in your branch. * Third, you should make sure your code will pass the build by running `./deploy` in the root directory. This will run tests, update `Ion.egg-info` and build the docs. At the end, you will probably get a message saying there are uncommitted changes. **This is fine**. As long as the script ends with an uncommitted changes message, you are ready to commit your code. {% hint style="info" %} In order to merge into master, it is recommended to write tests for any new code. If there is missing testing code, you should also write tests. To see some examples, look for `test.py` files in each app. More information can be found in the [Django docs](https://docs.djangoproject.com/en/dev/topics/testing/overview/). {% endhint %} * Fourth, commit your changes and write a simple and *descriptive* commit message. Messages like `fixes` are **NOT** descriptive. ## Scope The scope of your PR should be limited. You should not make PRs changing various unrelated portions of the code. For example, a PR focused on adding a new feature to printing should not be removing a test case for user authentication (although you are free to open a separate PR for that). In addition, you should not fix failed CI tests in the same PR (unless your code was the source of the errors). ## Opening a PR After pushing your proposed changes to your own personal fork, it is time for you to open a PR. * Head over to your fork and navigate to your branch via the drop down menu. At the top of your code, you should see a button called "Pull request". After you click it, you should see the opening of PR page. * On this page, ensure your head fork is set to the branch with your changes, while the base fork is set to the Ion repository's `dev` branch. {% hint style="warning" %} The *only* people should open PRs with the `master` branch as the base branch should be individuals with push access to at least the *dev* branch. These PRs against against the master branch should use the `dev` branch as the head fork. {% endhint %} * You must see a green check mark indicating that branch is able to merge. If it is not, you should rebase and make the branch mergeable before proceeding. * Write an informative title and describe your changes briefly in the description box. * Your changes should describe why you are changing something, how you are changing something, and what you are changing. * A good, informative PR description helps the maintainers review your changes faster. * Make sure only the changes you want to make are described in the PR. * Submit it! ## Responding to Reviews After your PR has been submitted, an Ion maintainer will review your PR. They can take three courses of action after reviewing: * approving the PR * closing the PR * or requesting changes to the PR If your PR has been approved, there is no need to take any further action unless requested to. The maintainer may wait for other maintainers to review your the code or may immediately merge the PR. If the PR has been closed, the reason is generally described by the closing maintainer. If a maintainer has requested changes, you should correct your code per the recommendations and push additional commits to your head branch (or just rebase your branch). # Style Guide The Ion code base generally follows the guidelines as set forth by [PEP8](https://www.python.org/dev/peps/pep-0008/). ## Main Points * Indent using 4 spaces. * Use underscores in favor of camel case for all names except the names of classes. * Limit all lines to a maximum of 79 characters. * Limit the line length of docstrings or comments to 72 characters. * Separate top-level functions and class definitions with two blank lines. * Separate method definitions inside a class with a single blank line. * Use two spaces before inline comment and one space between the pound sign and comment/ * Use a plugin for your text editor to check for/remind you of PEP8 conventions. * Capitalize and punctuate comments and git commit messages properly. ## Imports * Group imports in the following order: 1. Standard library imports 2. Imports from core Django 3. Related third-party imports 4. Local application or library specific imports, imports from Django apps * Avoid using `import *` * Explicitly import each module used #### Examples Standard library imports: ``` from math import sqrt from os.path import abspath ``` Core Django imports: ``` from django.db import models ``` Third-party app imports ``` from django_extensions.db.models import TimeStampedModel ``` Imports from your apps ``` from intranet.models import User ``` Explicit relative imports: Used to avoid hardcoding a module's package. This greatly improves portability. Use these when importing from another module in the current app. Absolute imports: Used when importing outside the current app. Implicit relative imports: Don't use these. Using them makes it very difficult to change the name of the app, reducing portability. Good: ``` from .models import SomeModel # explicit relative import from otherdjangoapp.models import OtherModel # absolute import ``` Bad: ``` from currentapp.models import MyModel # implicit relative import ``` ### References * [Google Python Style Guide](https://google.github.io/styleguide/pyguide.html). * [Google HTML/CSS Style Guide](https://google.github.io/styleguide/htmlcssguide.xml). * [Google Javascript Style Guide](https://google.github.io/styleguide/javascriptguide.xml). * [PEP8 Official Python Style Guide](https://www.python.org/dev/peps/pep-0008/). # Maintainer Workflow Maintainers of the Ion repository are responsible for the effective functioning of Ion. Generally, the Ion maintainers are the Ion leads for the years plus other senior Ion developers if applicable. ## Responsibilities Ion maintainers need to ensure that the code base remains good quality and nothing will break. Maintainers have the following responsibilities: * Triaging issues (labeling, closing issues as appropriate) * Reviewing pull requests * Maintaining our CI pipeline (run via Travis) * Ensuring the CI pipeline does not fail * Ensuring the code's compliance with the Ion Style Guide * Protecting Ion from security threats * Redacting private information from the Git repository * Determining the best course of action to take (being the final arbiters of decisions affecting Ion) ## Issues We have a system of labeling issues that are opened in the Ion repository. Examples of labels that we have include: * Browser compat * Bug * Cache * Client-side * Docs * Enhancement * Feature * Feedback * Help Wanted * Invalid * Question * Server-side * Vagrant * Upstream * Wontfix Issues out of the repository's scope should be closed. Duplicates of other issues should also be closed. Maintainers should however be respectful of people who open issues. ## Pull Requests Determining what code should be included in the Ion repository is a decision that lies ultimately with the Ion Lead. Together, with the Lead Sysadmins and the Faculty Sponsor, they serve as the FINAL decision makers on what to include or remove from Ion. Pull requests should be reviewed fairly regularly so that contributors can respond to the feedback that is given by the reviewers. The process for determining who should authorize/approve merges into the `dev` and `master` branches is to be determined each year by the Ion Lead. ## Evaluating Pull Requests When evaluating pull requests, maintainers look at various factors including: > * Correctness: Does the code do what it claims to do? Is the code correct in both the nominal case and the boundary cases? As a reviewer, this is your opportunity to point out edge conditions of which the original developer may not have been aware. > * Complexity: Does the code accomplish its task in a reasonably straightforward way? If you can point out simpler approaches that do not compromise the correctness or performance of the code, you should. > * Consistency: Does the code achieve its basic goals in a way that is consistent with how similar code in our codebase achieves those goals? Is it re-using the available libraries and utility classes? Where possible, has code been refactored for re-use instead of just copying and pasting? > * Maintainability: Could the code be extended by another developer on the team with a reasonable amount of effort? More than any item on the list, this is the karma investment you make by doing code reviews - the code you review today may be the code you have to update tomorrow, so taking the time to make sure it’s maintainable by others pays itself back to you. > * Scalability: Will the code be performant at the expected volumes? It is important that this question always be asked in the context of expected volumes. When building a new product in an untested market, it is fine to write code that works for 10,000 users but not 1M; if the product should be that successful, we will profile, optimize, and, when necessary, re-write the critical bits. The corollary is that we should not spend time optimizing code when the market demand is unproven. > * Style: Does the code match the team style guide? This should rarely be controversial. > > Adapted from PRs that add new features **should** include unit tests that cover all added code. # Repository Maintenance #### Branches We maintain two principal branches: * `master`: hosts the code for production; should be production-ready * `dev`: hosts the code in preparation for merge to `master` Feature branches can be created by any committer to host their feature or bugfix branch. Once a branch has gone three months without activity or it has been merged into the principal branches, it should be deleted and consider "stale". # Data Generation ## Generating Users and Eighth Periods Once you have your Vagrant environment set up, log into [GitLab](https://gitlab.tjhsst.edu/). If you do not have access to GitLab, email sysadmins\@tjhsst to request access. After you log in, clone the Ion Fake Data repo [here](https://gitlab.tjhsst.edu/sysadmins/web/ion/ion-fake-data). Follow the instructions in `README.md` to generate JSON files for importing user and eighth period activity data. Move the JSON files into the main directory of your local intranet copy, and ssh into your Vagrant machine. Then run `python manage.py import_users user_import.json` and `python manage.py import_eighth eighth_import.json` to import user and eighth period activities. In order to show eighth period activities on Ion, you need to create eighth period blocks. Run `python manage.py dev_create_blocks mm/dd/yyyy` and fill in an end date. This script will create blocks every Wednesday and Friday from the current date until the end date. Once blocks have been created, you can generate signups for the blocks on a specific date by running `python manage.py dev_generate_signups mm/dd/yyyy` by inputting a specific date which has eighth period blocks. The commands to generate blocks and signups for testing should ONLY be used in your local version of Ion. Now you have enough data to begin testing changes on users and eighth periods. # Production Steps to install Ion are described in our Ansible playbooks. Information on how to run Ion in production is located in [our runbooks](/general/documentation/runbooks). (Note that this information is closed to people with explicit permission only.) ## Troubleshooting tips 1. A 502 Bad Gateway ("Unable to contact an application server") indicates a problem with Daphne. 2. If you can access static files (like ) with no problem, but dynamic pages like the login page give errors or are very slow to load, it's almost definitely a problem with either Daphne or the database. (Conversely, if static files exhibit the same problem, it's probably an Nginx issue.) 1. You might be able to resolve Daphne issues by increasing the number of workers. Just make sure that 1) you add them to the Nginx config and 2) you increase the PostgreSQL connection limit appropriately (you'll want to run `systemctl restart postgresql && supervisorctl reread && supervisorctl update && systemctl restart nginx`when you've edited all the config files).\ Each Daphne worker seems to require about 75 connections, and you'll want a buffer over that. 3. If Ion is experiencing performance issues, try these troubleshooting steps: 1. Try and access a static file (see #2 above). 2. SSH to Ion and run `iotop`. If you see high disk usage, database access might be the slowdown. # User Experience # User Interface The user interface should be consistent and should not drastically impact the use experience. # Director The Director website management platform **Director 4.0** is a website management interface for student and activity websites. It was created during the 2019-20 school year and is currently used by many of the web application development classes. The application was created to provide a secure, beginner friendly, and highly customizable website hosting platform. To achieve this goal, many features, including a web terminal and online editor, were implemented. The project has been completely [student-built](https://director.tjhsst.edu/about), and was the senior research project of Class of 2020 Sysadmins. Director 4.0 was the next iteration of web3, the senior research project of Eric Wang (class of 2017). See what's new in Director 4.0 [here](https://director.tjhsst.edu/docs/whatsnew-director4/). The point of contact for Director is [the Director Lead.](/general/sysadmins-list#current-leads) ## Technologies Used * Python * Django * Docker ## External Links * [Director](https://director.tjhsst.edu/) * [Director Guide](https://director.tjhsst.edu/docs/) * [Director on GitHub](https://github.com/tjcsl/director) # Development # Vagrant Setup Director Development using Vagrant ## Getting Started Similar to Ion, Director has a setup using Vagrant. This page will go over about setting up for Director using Vagrant. Refer to the Github page for setting up a developer environment for more up to date information. First off, install [VirtualBox](https://www.virtualbox.org/wiki/Downloads) and [Vagrant](https://www.vagrantup.com/downloads), VirtualBox is a virtualization program, and Vagrant is the development environment in where we'll run Director. For more information about those two services, please take a look at their documentation linked below. If it is not installed already, install [Git](https://git-scm.com/). Git is a free and open source distributed version control system designed to handle everything from small to very large projects with speed and efficiency. This is essential for Director if you are going to be pushing changes to it. Ensure you have an SSH key set up with GitHub by running `ssh -T git@github.com`. You should be greeted by your username. If not, set up an SSH key with GitHub by following [these instructions](https://help.github.com/articles/generating-an-ssh-key/). Clone the Director 4.0 repository onto your computer and `cd` into the new directory. Essentially just run `git clone git@github.com:tjcsl/director4.git director && cd director`. {% hint style="info" %} Note: if your host machine is running Windows, please run `git config core.autocrlf input` before cloning to prevent line ending issues. {% endhint %} ```bash $ git clone git@github.com:tjcsl/director4.git director $ cd director ``` Once inside the `director` directory, run `vagrant plugin install vagrant-vbguest`. If you are on Windows, also run `vagrant plugin install vagrant-winnfsd`. Run `vagrant up && vagrant reload` and wait while the development environment is set up. This will download a Vagrant image and provision the resulting VM. ## Post-Setup After successfully setting up the Vagrant environment, you will want to actually access your sandbox. Start by connecting to the Vagrant box using `vagrant ssh` to connect to the VM. Once inside, run `cd director` to change into the repo and `./scripts/install_dependencies.sh` to install Director's Python dependencies using `pipenv`. Once completed, you may now work on Director `scripts/start-servers.sh` will open a `tmux` session with the four servers each running in a separate pane. * Note: If you are not familiar with `tmux`, we recommend and as starting resources. * See [this](https://github.com/tjcsl/director4/blob/master/docs/docs/tmux.md) for an explanation of the components of the development `tmux` When you are finished, type `exit` to exit the VM and `vagrant halt` to stop it. When you want to work on Director 4.0 again, `cd` into this directory, run `vagrant up` and `vagrant ssh` to launch the VM and connect to it, and then run `exit` and `vagrant halt` to exit and shut it down. ## Documentation * VirtualBox - * Vagrant - * Git - # PR Workflow Follows Ion's [PR workflow](/services/ion/development/pr-workflow) # Style Guide Follows Ion's [style guide](/services/ion/development/style-guide) # Maintainer Workflow Maintainers of the Director repository have the same responsibilities as outlined in Ion's [maintainer workflow](/services/ion/development/maintainer-workflow) # Production Information on production Director is located in [our runbooks](/general/documentation/runbooks). # Workstations CSL desktop workstations In the CSL, **workstations** are a group of computers that are maintained by the Sysadmins to provide TJ students and staff a Linux environment. Currently there are over 60 **HP Z240 Tower Workstations** that are located in rooms 200 and 202. These workstations are often used by computer science classes which includes Computer Vision and Artificial Intelligence. Seniors, who are enrolled in doing research for the Syslab or Web/Mobile App Development, often use our workstations as well. ## Technical Specifications Most workstations have **similar** specifications listed below: | Specification | Description | | ------------- | --------------------------------------- | | Motherboard | HP 802F | | CPU | Intel(R) Core(TM) i7-8700 CPU @ 3.40GHz | | RAM | 1-2x Hynix 8GB DDR4-2133 | | GPU | NVIDIA Quadro K620 | | Hard Disks | 1 Terabyte HDD | | OS | Debian 13 | ## The Pit Workstations

The Pit - June 2023

**The Pit Workstations** are workstations located in the Pit that are mainly intended for Sysadmin use. Some of the workstations below don't have the same specifications that are listed above. * **`mozart`** - Runs Ubuntu 22.04, has the same specifications as other workstations. * **`turing`** - Runs Debian 12, has the same specifications as other workstations. ## Room 200 Almost half of our workstations are located in Rm 200. These workstations are used mainly for AI and senior research. * `eevee` * `meowth` * `psyduck` * `growlithe` * `machop` * `geodude` * `slowpoke` * `bulbasaur` * `ivysaur` * `venusaur` * `charmander` * `charmeleon` * `charizard` * `squirtle` * `wartortle` * `blastoise` * `caterpie` * `metapod` * `butterfree` * `pikachu` * `gastly` * `beedrill` * `pidgey` * `snorlax` * `onix` * `cubone` * `lapras` ## Room 202 The other half of our workstations are located in Room 202. These workstations can be accessed remotely through Guacamole, or used over ssh / in-person. * `prokofiev` * `hindemith` * `puccini` * `rachmaninoff` * `wagner` * `vivaldi` * `stravinsky` * `schumann` * `handel` * `gershwin` * `faure` * `debussy` * `copland` * `chopin` * `brahms` * `bernstein` * `satie` * `ravel` * `scarlatti` * `schubert` * `berlioz` * `bach` * `haydn` * `tchaikovsky` * `mendelssohn` * `pachelbel` * `mahler` * `vecchio` * `holst` * `liszt` ## Accessing the Workstations Accessing is fairly simple when it comes to workstations. If you're on VPN, just running `ssh .csl.tjhsst.edu` (running `ssh ` should also work as well) should do the trick, however if your not on VPN, there are a few steps: {% hint style="info" %} Note that you should always check if the workstation is SSHable in the first place, by simply checking if it is pingable by running `ping `. If it is not pingable, then it is likely off or broken. Ask the Workstations Lead(s) if you have questions. {% endhint %} 1. Open the terminal and run `ssh .remote.tjhsst.edu`. This is our remote access servers, which allows you to access any CSL-related items outside of TJ. This should open a prompt that would asks you to type in your password. 2. Once your in, you should just SSH into the workstation as so: `ssh `. For more specific information on CSL workstations are located in [our runbooks](/general/documentation/runbooks). Note that some workstations are being used for other purposes and may not be accessible. # Signage The CSL Signage displays **Signage** refers to the set of ten electronic displays located all over the school. A "Signage" refers to a display that displays useful information maintained by the CSL across the school. All signages (except for `cs-audlob`) have the same types of features which include: * The schedule of the current school day. * Ion announcements * 8th period activities. * A bus map * A map of the whole school. Signages can also showcase important announcements made by the Sysadmins or FCPS. The current version of Signage is **Signage3**. The core of Signage runs on Intel Compute Sticks ([official website](https://www.intel.com/content/www/us/en/products/boards-kits/compute-stick.html)) running Ubuntu 16.04 LTS. The main contact for signages are our Signage Lead(s). The Intranet Lead(s) are usually the deputies for signages if the Signage Lead(s) are not present. ## History ### Signage1 The first Signage displays were installed around the first release of Ion ([source](https://tjhsst.edu/~jwoglom/ion.pdf)) (around May 2016). They were originally developed as a complement to Ion. These efforts were led by James Woglom. The Signage displays ran on a mixture of Raspberry Pi 1s, Raspberry Pi 2s, and Raspberry Pi 3s. ![](https://3496721629-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-LKalf7rYxXVbB0kIEA2%2F-LLS90aJVn69Kps-Gfq3%2F-LLS8ubkuQrK74gRD89c%2Fsignage1.png?alt=media\&token=97e5c283-3884-4a79-8d7a-8fad8fd421b2) ### Signage2 Development of Signage2 began in April of 2017 to serve as a rewrite of Signage1 using Python's Kivy framework. Various limitations imposed by Kivy and slow performance led to the end of development around March of 2018. ### Signage 3 Development of Signage3 began in March of 2018 to serve as a rewrite of Signage as a webpage hosted on Ion. Signage3 was deployed on newly-arrived Intel Compute Sticks on April 12, 2018. Signage3 remains the current Signage deployed throughout the school. ## List The current deployed Signage displays are named (and located): * `cs-nobel` (Nobel commons) * `cs-curie` (Curie commons) * `cs-galileo` (Galileo commons) * `cs-cafeteria` (Cafeteria) * `cs-gandhi-a` (Ghandi commons) * `cs-audlob` (Outside the audlob) * `cs-library` (Library hallway) * `cs-hopper` (Hopper commons) * `cs-cafe` (Cafe commons) * `cs-einstein-a` (Einstein commons) ## Current Setup Each Signage display has their own Intel Compute Stick which runs Ubuntu Server 20.04 LTS. Using getty, each Signage display logs in as `user` and opens up a specific web page on Ion (`https://ion.tjhsst.edu/signage/display/`) in Chromium. The Signage pages can be rendered server-side or as iframes. The code for Signage can be found [here](https://github.com/tjcsl/ion/tree/master/intranet/apps/signage). # Setup ## Network Configuration Overview Each Signage Intel Compute Stick connects to TJ's Windows network. Each stick's MAC address is whitelisted by TJ's Tech Team to access the `tjhsst` network. Using [wpa\_supplicant](https://wiki.archlinux.org/index.php/WPA_supplicant) and [systemd-networkd](https://wiki.archlinux.org/index.php/Systemd-networkd), each stick is assigned an IP on the Windows network. ## Installation In the past, Signage was a separate web application from Ion, viewed on Raspberry Pis. Now, Signage is run on Intel Compute Sticks running Ubuntu Server with a minimal GUI. This guide will show you how to set up a new stick. ### Manual Stuff #### Installing the OS When you first get one of the [Compute Sticks](https://livedoc.tjhsst.edu/wiki/Compute_Sticks), it should come pre-loaded with Windows 10 by default. If this is not the case, you probably got the wrong version of Compute Stick (the Ubuntu ones aren't powerful enough). **Prepare the Installation USBs** These steps only have to be done if there are no installation USBs around. *This guide assumes (hopefully correctly) that you know how to manage USBs on whatever computer you are using.* 1. Download the OS. * Head on over to Ubuntu's [website](https://ubuntu.com/download/server) and download the `.iso` for the the latest LTS version. 2. Put the OS on a USB. * On Linux or Mac, use the `dd` utility to flash a USB with the OS installer. On Windows, use [Win32DiskImager](https://sourceforge.net/projects/win32diskimager/::) or something to that nature. * Example `dd` command: `sudo dd if=ubuntu_server-16.04.iso of=/dev/sdx progress=status`, replacing `sdx` with the actual USB device identifier. 3. Download the `wpa_supplicant` package and dependencies. * On an Ubuntu system with the same architecture as the Compute Stick (preferably an existing Compute Stick), use this command to download `wpa_supplicant` and all of its dependencies: ``` apt-get download wpa_supplicant && apt-cache depends -i wpa_supplicant | awk '/Depends:/ {print $2}' | xargs apt-get download ``` * On a Windows or Mac system, you will have to do this manually by tracing dependencies. **Get Configuration Files** There is a GitLab [repo](https://gitlab.tjhsst.edu/signage3/cs-config) which contains important scripts and config files to setup networking and an Ansible-ready system. You should clone it by running within the mounted wpa\_supplicant drive ``` git clone git@gitlab.tjhsst.edu:signage3/cs-config.git ``` **Boot from the OS Install Disk** 1. Plug in a keyboard into the Compute Stick. 2. Plug the Stick into an available monitor. 3. Turn the Stick on. 4. As the Stick is booting, repeatedly press the F10 key until you see a boot menu. * If this does not work, reboot and try again. 5. At the boot screen, select the USB as the boot device. * If you can't boot, [Google](https://google.com) what went wrong. 6. Follow the on-screen prompts to install Ubuntu Server LTS. * Try to install as few extra features as possible. * Set the username/password as instructed by the Signage Lead. The password, however, will be changed later. **Mount the USB** Now, plug in the other installation USB (the one with the packages on it) into the Compute Stick. Mount it with ``` sudo mount /dev/sdxn /mnt ``` where `sdxn` is the USB partition you stored the files on. `x` will usually be `a`, and `n` will usually be `1`. ### Automatic Configuration Scripts to complete initial setup of the script can be found on [GitLab](https://gitlab.tjhsst.edu/signage3/cs-config). You will need to copy this repository to a flash drive. * Cd into `/mnt/cs-config`. * Run `./setup_network.sh ` to configure networking. * Reboot. * Check that you have internet (`ping`) and use `ip a` to check that you have been assigned an IP address. * Run `./setup_finish.sh` to complete setup (install python basic config). * From your device, run the CSL Ansible play for signage (`signage.yml`) to configure the base of the system. # Administration ## Sign/Page Administration ### Develop New Page Views for each Signage page are located in the [pages.py code](https://github.com/tjcsl/ion/blob/master/intranet/apps/signage/pages.py). To add a new page, define the context in `pages.py`. The template should be defined in `templates/signage/pages/.html`. ### Create New Page Each Signage display renders multiple pages. Each page is defined within a `Page` model. Within Django admin, define the: * name: Some descriptive name to represent the Page (required) * template: The location of the template (`templates/signage/pages/