Category Archives: System Maintenance

Getting ElasticSearch Node Versions

Kibana doesn’t tell you what version your nodes are currently running which can be frustrating if you get distracted during an upgrade process. Here is a simple PowerShell script that gets all the nodes and their current version.

$cred = Get-Credential
$nodes_raw = Invoke-RestMethod -Method Get -Uri "http://elasticsearch:9200/_nodes" -Credential $cred

#get the names of each node
$node_names = $nodes_raw.nodes | Get-Member -MemberType NoteProperty | Select -ExpandProperty Name

#convert from object properties to an array of objects
$nodes = foreach ($node_name in $node_names) {
    $nodes_raw.nodes."$node_name"   
}

# Select desired info about the node
$nodes | select name, version
Advertisement

Use PowerShell to export AD User accounts to a CSV with their properties flattened

This queries for specific users (modify this to match your need)

Taking a list of users Flatten-Object will go through their properties and join multivalued attributes with a semi-colon so that you can export it in a useful way to CSV.

# Change this to and LDAP filter that fits your needs
$admins = Get-ADUser -filter "employeeType -eq 'Administrator Acct'" -Properties *

# take multivalued properties and join them with ;
function Flatten-Object {
    [CmdletBinding()]
    param(
        [Parameter(ValueFromPipeline)]
        $object
    )
    process {
        $flattened_object = [pscustomobject]@{}

        foreach ($field in $object | get-member) {
            if ($field.MemberType -eq “Property” -and $field.Name -notlike “__*”) {
                $flattened_object | Add-Member -NotePropertyName $field.Name -NotePropertyValue $($object.$($field.Name) -join ";")
            }
        }
        $flattened_object
    }
}

$admins | Flatten-Object | Export-Csv -Path "user.csv" -NoTypeInformation

Prettify/Format LDAP Queries using Powershell

Here is a simple script to indent add new lines to a PowerShell query.

$ldap = "(&(&(objectCategory=person)(objectClass=user))(lastLogonTimestamp<=128752108510000000)(!(userAccountControl:1.2.840.113556.1.4.803:=2)))"

$ldap = $ldap -replace '\(([\&\|\!]+)', "(`$1`n"
$ldap = $ldap.replace(')', ")`n")

$lines = $ldap.split("`n")
$indent = 0
$new_lines = ForEach ($line in $lines) {
    if  ($line -eq ")") {
        $indent--
    }

    ("`t" * $indent) + $line

    if ($line -eq "(&" -or $line -eq "(|" -or $line -eq "(!") {
        $indent++
    }
}

$new_lines

Results in a nicely formatted query:

(&
	(&
		(objectCategory=person)
		(objectClass=user)
	)
	(lastLogonTimestamp<=128752108510000000)
	(!
		(userAccountControl:1.2.840.113556.1.4.803:=2)
	)
)

Dynamically Getting AD User Properties

Sometimes you want to return properties that match a specific pattern. The following will do that for you:

Get-ADUser Administrator -Properties * | %{Get-Aduser $_.sAMAccountName -Properties @($_ | Get-Member -MemberType Property | Select -ExpandProperty Name | Where {$_ -like "ms*"})}

This gets a user with all properties, then filters those properties by the final like statement and re-gets the user with just those properties. You can also do it with a single get-aduser:

Get-ADUser Administrator -Properties * | %{$_ | Select -Property @($_ | Get-Member -MemberType Property | Select -ExpandProperty Name | Where {$_ -like "ms*"})}

The only downside to this is it doesn’t include the default attributes, so it can be a bit less usefull. If you want to include some specific attributes + all matching a like statement then you can do the following:

Get-ADUser Administrator -Properties * | %{$_ | Select -Property @(@("Name","samACcountName", "Department", "Description", "DisplayName", "DistinguishedName", "employeeType") + ($_ | Get-Member -MemberType Property | Select -ExpandProperty Name | Where {$_ -like "ms*"}))}

And finaly as a code block instead of a one liner:

$user = Get-ADUser Administrator -Properties * 
$msProperties = $user | Get-Member -MemberType Property | Select -ExpandProperty Name | Where {$_ -like "ms*"}
$user | Select -Property @(@("Department", "Description", "DisplayName", "DistinguishedName", "employeeType") + $msProperties)

Reducing the number of replicas in Elasticsearch

Sometimes you just want to run a single Elasticsearch node and not have it constantly alert that it has no were to write its replicas. Since Elasticsearch and more templates default to at least 1 replica we have to make changes to Elasticsearch and to the templates. First change the default:

curl -XPUT 'localhost:9200/_settings' -d '
{
    "index" : {
        "number_of_replicas" : 0
    }
}
'

Then we can list all the templates and figure out which ones need updates as well

curl -XGET -H 'Content-Type: application/json' 'localhost:9200/_template/*?pretty'

Then for each of the templates you want to update. For instance to update a Logstash template use the following:

curl -XPUT -H 'Content-Type: application/json' 'localhost:9200/logstash-*/_settings' -d '{ "number_of_replicas" : 0 } }'

Python OAuth2 auth and bearer token caching

To access many APIs you need to use OAuth2, sending a client id and secret to an endpoint to get a token back. Then send that token with future calls as authentication.

The particular API I was calling would also return a number of seconds the bearer token would be good for.

Hopefully this code will help jumpstart someone else along the way to using python and APIs.

#!/usr/bin/env python3

import requests
import json
import datetime

import os.path
from os import path

cache_file = "/var/tmp/oauth.json"
client_id = '**your client_id here**'
client_secret = '**your secret here**'

def getToken():
    url = "https://api.wherever.com/oauth2/token"

    data = {
      'client_id': client_id,
      'client_secret': client_secret,
      'grant_type': 'client_credentials'
    }

    response = requests.post(url, data=data)

    data = json.loads(response.text)
    expiration_time =  datetime.datetime.now() +  datetime.timedelta(seconds=data['expires_in'])
    data['expiration_date'] =   int(expiration_time.utcnow().timestamp())

    with open(cache_file, "w") as outfile:
        json.dump(data, outfile)
    return data



if path.exists(cache_file):
    #Reading cache
    with open(cache_file, "r") as infile:
        access_token = json.load(infile)
    if int(datetime.datetime.now().timestamp()) > access_token['expiration_date']:
        #Token expired, get new
        acces_token = getToken()
else:
    #No cached value, get and cache
    access_token = getToken()


bearer_token = access_token["access_token"]
headers = {
        'Authorization': f'bearer {bearer_token}'
}

#The rest of the requests go here and pass that header

Use PowerShell to find what process is using a port

Sometimes you just need to know what ports a process is listening on, or what process is listening on a port…so here you go, just replace the process or port and have at it.

$process = "svcHost"
Get-NetTCPConnection | Where OwningProcess -in (Get-Process | Where ProcessName -eq $process | Select -ExpandProperty Id)
$port = 135
Get-Process -Id (Get-NetTCPConnection -LocalPort $port).OwningProcess

Steps for Setting up Rancid + CentOS + Git

Setting up Rancid so that it pushes config up into a Git repository. This way you have Rancid getting the config when it changes, and Git storing a history of those changes.

Install CentOS & update

# install pre-requisites
sudo yum install wget gcc perl tcl expect
#setup needed groups and users
sudo groupadd netadm
sudo useradd -g netadm -c "Networking Backups" -d /home/rancid rancid

#working directory to store source
sudo mkdir /home/rancid/tar

#download source and extract
sudo su
cd /home/rancid/tar
wget ftp://ftp.shrubbery.net/pub/rancid/rancid-3.9.tar.gz
tar -zxvf rancid-3.9.tar.gz

# configure/make and install
cd ./rancid-3.9
./configure --prefix=/usr/local/rancid
make install

# copy template config and set file permissions
chmod 0640 /home/rancid/.cloginrc
chown -R rancid:netadm /home/rancid/.cloginrc
chown -R rancid:netadm /usr/local/rancid/
chmod 775 /usr/local/rancid/

Configure Git

  1. yum install git
  2. Modify /usr/local/rancid/etc/rancid.conf
  3. Change RCSSYS=git
  4. Save and close
  5. Switch to rancid user su - rancid
  6. Create SSH key ssh-keygen -o -t rsa -b 4096 -C "email@example.com"
  7. Copy the key to insert into github/gitlab `vim ~/.ssh/id_rsa.pub`
  8. Setup Defaults
    1. git config --global user.name "Rancid" git config --global user.email "email@example.com"
  9. Configure Rancid Groups
  10. Build initial group folders `​/usr/local/rancid/bin/rancid-cvs`
  11. Go to the device group folder cd /usr/local/rancid/var
  12. For each Device group
  13. cd {device group} git remote rename origin old-origin git remote add origin git@{git server}:{repo}/{git project}.git git push -u origin --all
  14. Setup hook to push to the git server. In each device group
    1. Open the  post-commit hoof file
vim .git/hooks/post-commit
#!/bin/sh
# push the local repo to the remote server on commit
git push origin

Finding who is using a Windows Share in PowerShell

You can find the users and computers connected to a share pretty easily in powershell.

$share_name = "*"
$shares = Get-WmiObject Win32_ServerConnection 
$shares = $shares | Where-Object {$_.ShareName -like $share_name} | `
                    Select-Object ShareName, UserName, `
                                  @{n="Computer";e={[System.Net.Dns]::GetHostEntry($_.ComputerName).HostName}}
$shares | Group-Object -Property ShareName | ` 
          Select Name, `
                 @{n="Computers";e={$_.Group | Select -ExpandProperty Computer | Get-Unique}},`
                 @{n="Users";e={$_.Group | Select -ExpandProperty UserName | Get-Unique}},`
                 Group

$share_name is a filter to narrow down the shares you are checking. Other than that it gets all the share connections then groups them by share. It shows unique users and computers connected to each share.

SCCM PXE Booting with NSX-T

Symptoms:

PXE boot stuck at “Waiting for Approval”

If you virtualize a Distribution Point in SCCM and want to use it for PXE booting you need to make some changes in NSX-T in order to allow it to work.

PXE booting sends a DHCPREQUEST directly to your DPS. By default NSX-T blocks servers from getting or replying with DHCP requests.

To enable it for specific servers create a “Segment Security Policy” profile in NSX-T and disable DHCP Server Block

Create a new network segment with the correct VLAN and assign the policy you just created to it.

In VCenter move the DPs to the newly created segment.