Hari Sekhon - Knowledge Base from 20 years in DevOps, Linux, Cloud, Big Data, Security, AWS, GCP etc

Repo focused on advanced knowledge, tips & tricks, debugging & fixes.

I'm gradually migrating my huge private personal knowledge base into this public repo.

Hit the watch button on the top right for updates.

Some of these pages are still in progress pending porting and populated yet from my private knowledge base pages, some are stubs pending migration.

Much more advanced knowledge is captured in code and code comments. References to my major GitHub Repos are provided throughout where relevant.

References to official docs and some classic books are provided for those who want to thoroughly cover the basics.

Index

• GitHub Repos
• Systems
  • Operating Systems
  • Core Services - DNS, DHCP, SSH, FTP
  • Editors
  • Terminal Multiplexers
• Architecture & Documentation
• Git
• Cloud
• Containerization
  • Docker
  • Kubernetes
    • Kubernetes Core
    • Cloud Hosted Kubernetes
    • On-premise Kubernetes
    • Local Kubernetes Dev
    • Kubernetes Applications & Lifecycle Management
  • Container Registries
  • Container Security
• Programming
  • Languages
  • Build Systems
  • Artifact Registries
  • Code Scanning
• Infrastructure-as-Code
• Configuration Management
• CI/CD
• Internet Services
• Databases / RDBMS
  • MPP Databases
  • Hosted MPP Databases
  • Database Management
• NoSQL
  • Caching
  • Search
• Coordination & Discovery Services
• Pub / Sub & Message Queues
• Data
  • Data Formats
  • Hadoop Ecosystem
  • Data Science & Machine Learning
  • Time Series Databases
  • Visualization
• Security
• Virtualization
• Hardware
• Networking
  • Load Balancing
• Monitoring
  • Performance Engineering
• AI & Automation
• Work
  • Training & Interview Materials
  • Banking Industry
  • Travel
• Miscellaneous
  • Dev / DevOps / Open Source
  • Vendors
  • Multimedia
  • Marketing
  • Crypto
  • Old Docs
• Star History
• Memes
• More Core Repos
  • Knowledge
  • DevOps Code
  • Containerization
  • CI/CD
  • DBA - SQL
  • DevOps Reloaded
  • Templates
  • Misc

This is the lost documentation.

GitHub Repos

Full of open source tools and scripts used throughout the rest of these docs to make it easier to install and operate these technologies:

• DevOps-Bash-tools - 1000+ Bash scripts, advanced shell environment customizations, quick installation scripts and dot configs
• DevOps-Python-tools - 80+ Python programs for DevOps & Data
• DevOps-Perl-tools - 25+ Perl programs for DevOps
• DevOps-Golang-tools - a handful of Golang programs
• Kubernetes-configs -
  many Kubernetes applications and advanced templates with Tips & Tricks from production environments
• SQL-scripts - 100+ database scripts
• Templates - templates for many common file formats for programming languages, build systems and DevOps configs
• Diagrams-as-Code - Cloud & Open Source architecture diagrams

Systems

Operating Systems

• Linux
  • Debian
  • Ubuntu
  • Redhat
  • Alpine
  • Gentoo
• Windows
• Mac
  • Homebrew
• Shell
  • Bash
  • direnv / .envrc
  • Advanced & Third Party Commands
• Disk Management
• Logging
  • Syslog-ng

Core Services - DNS, DHCP, SSH, FTP

• SSH
  • SSH Tunnelling
• FTP
• DNS
• DHCP

Editors

• Editors & IDEs
  • Vim
  • IntelliJ
  • VS Code
• Coder

Terminal Multiplexers

• Screen
• Tmux

Architecture & Documentation

• Architecture TODO
• Product Selection TODO
• Diagrams-as-Code
  • D2 lang
  • MermaidJS
• Documentation
  • Markdown
  • MkDocs
• Microservices TODO
• Medium.com (blog)
• Systems Design

Git

• Git
  • Git Workflow Branching Strategies
• Git Repo Hosting Providers:
  • GitHub TODO
  • GitLab
  • Azure DevOps TODO
  • Bitbucket TODO
• Pre-Commit

Cloud

• Cloud
• Cloud Platforms:
  • AWS
  • GCP
  • Azure
  • Digital Ocean
• Third Party Tools:
  • MinIO
  • HashiCorp Vault
• PaaS

Containerization

Docker

• Docker
  • Dockerfile TODO
  • Docker Compose
  • Docker Swarm
• Podman
• Dockerfiles for many technologies

Kubernetes

Kubernetes Core

• Kubernetes
• Kubernetes Upgrades
• Kubernetes Configs
• Kubernetes Production Ready Checklist
• Traefik
• Kong API Gateway
• Istio TODO
  • Kiali
• Kubeflow TODO

Cloud Hosted Kubernetes

• AWS EKS TODO
• GKE TODO
• AKS TODO
• Karpenter

On-premise Kubernetes

• K3s
• Rancher
• RKE2
• Portworx

Local Kubernetes Dev

• Docker Desktop embedded Kubernetes (easiest to use)
• MiniKube
• MiniShift - for OpenShift upstream okd
• K3d
• Kind

Kubernetes Applications & Lifecycle Management

• Helm
• Kustomize
  • Helm vs Kustomize
• ArgoCD
• Cert Manager
• Kubernetes Configs

Container Registries

• DockerHub
• AWS ECR
• GCP GAR
  • GCP GCR (deprecated / shut down)
• Azure ACR
• GitHub GHCR
• Redhat Quay.io (there is an on-prem installable too)
• JFrog Container Registry
• Nexus
• Harbor
• Chart Museum

Container Security

• Container Scanning
• Trivy
• Grype
• Clair

Programming

• Programming
• Binaries Debugging

Languages

• Perl
• Python
  • Anaconda
• Golang TODO
• Ruby
• R TODO
• TCL / Expect
• JVM:
  • Java TODO
  • JDBC
  • JVM Performance Tuning
  • Groovy
  • Scala TODO
  • Jython
  • JRuby
  • log4j
• APIs
• Regular Expressions
• Testing

Build Systems

• Build Systems
• Make
• Maven
• Gradle
• SBT

Artifact Registries

• Artifact Registries
• Artifactory
• Nexus TODO

Code Scanning

• Code Scanning
  • Trivy
  • Grype
• Code Quality
• SonarQube
• CodeClimate
• Codefresh

Infrastructure-as-Code

• Terraform
  • tfenv
  • Terragrunt
    • tgswitch
  • Atlantis
• Pulumi

Configuration Management

• Ansible
• Puppet TODO
• YAML

CI/CD

• CI/CD
• Self-Hosted CI/CD:
  • Jenkins
    • Jenkins-on-Kubernetes
    • HariSekhon/Jenkins - master Jenkinsfile template and Groovy Shared Library functions
  • TeamCity TODO
    • HariSekhon/TeamCity-CI
  • Concourse
  • GoCD TODO
• Git Repo Providers hosted CI/CD:
  • GitHub Actions
  • GitLab CI TODO
  • Azure DevOps TODO
  • Bitbucket Pipelines TODO
• 3rd Party hosted CI/CD:
  • CircleCI
  • TravisCI
  • Drone.io

Internet Services

• Web
• CDNs
  • Cloudflare
• HTTP Proxying
• APIs

Databases / RDBMS

• SQL TODO
• SQL scripts
• Databases
• Data Warehousing TODO
• MySQL TODO
• PostgreSQL TODO
• Oracle
• CloudBeaver

MPP Databases

Massively Parallel Processing databases - horizontally scaling databases:

• Vertica
• Greenplum
• Impala TODO
• Presto
• Apache Drill

Hosted MPP Databases

• Distributed SQL
• Snowflake
• Google BigQuery TODO
• AWS Redshift TODO

See also the Hadoop Ecosystem section further down for more distributed SQL databases.

Database Management

• LiquiBase TODO
• AtlasGo
• ByteBase TODO

NoSQL

• Cassandra TODO
• Couchbase
• HBase TODO
• MongoDB TODO
• Neo4j

Caching

• Redis TODO
• Memcached
• Couchbase
• Oracle Coherence

Search

• Search
• Elasticsearch
  • Kibana
• Solr
• Logstash TODO

Coordination & Discovery Services

• Consul
• Etcd
• ZooKeeper

Pub / Sub & Message Queues

• Pub / Sub & Message Queues
• Kafka
• RabbitMQ
• Celery
• AWS Kinesis

Data

• Data
• Big Data
• Data Warehousing TODO
• Spark TODO
• Airflow TODO
• Apache Camel
• JSON
• Informatica

Data Formats

• Data Formats
• Avro
• Parquet
• JSON

Hadoop Ecosystem

• Spark TODO
• Kafka
• Hadoop TODO
  • Kerberos Hadoop TODO
  • YARN TODO
  • HDFS TODO
• Pig TODO
• Hive TODO
  • Sqoop
• Impala TODO
• Presto
• Apache Drill
• Kudu
• Kylin
• Storm TODO
• Mahout
• ZooKeeper

Data Science & Machine Learning

• Data Science TODO
• Machine Learning TODO
• Python
  • Anaconda TODO
• R TODO
• Octave TODO

Time Series Databases

• OpenTSDB
  • TCollector
• T.I.C.K. platform:
  • Telegraf TODO
  • InfluxDB
  • Chronograf TODO
  • Kapacitor

Visualization

• Visualization
• Tableau

Security

• Security TODO
• SSL TODO
• Kerberos TODO
• FreeIPA TODO
• Keycloak
• Penetration Testing TODO
• Secrets Managers
• SOC - Security Operations Center

Virtualization

• Virtualization
• VirtualBox
• Vagrant
• Packer
• OpenStack TODO
• KVM

Hardware

• Hardware
• Storage

Networking

• Networking TODO
• Firewalls
• VPNs
• Cisco TODO
• Juniper TODO
• Foundry TODO
• Keepalived

Load Balancing

• Load Balancing TODO
• HAProxy
• F5

Monitoring

• Monitoring
• Nagios
• Prometheus TODO
• Thanos TODO
• Grafana TODO
• Notifications
  • OpsGenie
  • PagerDuty TODO
• Fluentd

Performance Engineering

• Performance Engineering
• Stress Testing

AI & Automation

• AI
• Automation

Work

Training & Interview Materials

• Training
• Interview Materials

Banking Industry

• Banking Industry

Travel

• Travel

Miscellaneous

Dev / DevOps / Open Source

• DevOps Misc
• Dev Misc
• Open Source
• IRC

Vendors

• Hashicorp
• Confluent
• Atlassian

Multimedia

• Multimedia

Marketing

• Marketing

Crypto

This section is probably the most financially important thing you'll ever learn.

• Bitcoin
  • more financially beneficial than your Tech knowledge
  • the most important knowledge along with avoiding
    feminist divorce law
• Crypto
  • every day you're not in crypto you're losing money due to constant inflationary money printing by central banks and
    governments to fund budgetary deficits caused by universal suffrage spending liabilities
  • even many investments, which are also taxed and often confiscated by governments or ex-wives, are losing ground to
    inflation in real terms

Old Docs

• Hazelcast
• Mesos

Star History

Memes

From my LinkedIn:

I tried to make these docs as short as possible... but as you can see above, I didn't quite make it.

I've been both the bottom and top of this meme:

More Core Repos

Knowledge

DevOps Code

Containerization

CI/CD

DBA - SQL

DevOps Reloaded

Templates

Misc

The rest of my original source repos are here.

Pre-built Docker images are available on my DockerHub.