Wednesday, 1 March 2017

Puppet-bitbucket for Cloud

puppet-bitbucket


Table of Contents

  1. Overview
  2. Module Description - What the module does and why it is useful
  3. Setup - The basics of getting started with Bitbucket
  4. Usage - Configuration options and additional functionality
  5. Reference - An under-the-hood peek at what the module is doing and how
  6. Limitations - OS compatibility, etc.
  7. Development - Guide for contributing to the module
  8. Testing - How to test the Bitbucket module
  9. Contributors

Overview

This is a puppet module to install Atlassian Bitbucket. On-premises source code management for Git that's secure, fast, and enterprise grade.
Module VersionSupported Stash/Bitbucket versions
2.1.x4.5.x - 4.6.x
2.0.x4.0.x - 4.4.1

Module Description

This module installs/upgrades Atlassian's Enterprise source code management tool. The Bitbucket module also manages the bitbucket configuration files with Puppet.

Setup

Bitbucket Prerequisites

  • Bitbucket requires a Java Developers Kit (JDK) or Java Run-time Environment (JRE) platform to be installed on your server's operating system. Oracle JDK / JRE (formerly Sun JDK / JRE) versions 7 and 8 and Open JDK/ JRE versions 7 and 8 are currently supported by Atlassian.
  • Bitbucket requires a relational database to store its configuration data. This module currently supports PostgreSQL 8.4 to 9.x and MySQL 5.x. We suggest using puppetlabs-postgresql/puppetlabs-mysql modules to configure/manage the database. The module uses PostgreSQL as a default.
  • Whilst not required, for production use we recommend using nginx/apache as a reverse proxy to Bitbucket. We suggest using the jfryman/nginx puppet module.

What Bitbucket affects

If installing to an existing Bitbucket instance, it is your responsibility to backup your database. We also recommend that you backup your Bitbucket home directory and that you align your current Bitbucket version with the version you intend to use with puppet Bitbucket module.
You must have your database setup with the account user that Bitbucket will use. This can be done using the puppetlabs-postgresql and puppetlabs-mysql modules. The mysql java connector can be installed using the puppet/mysql_java_connector module.
When using this module to upgrade Bitbucket, please make sure you have a database/Bitbucket home backup. We plan to include a class for backing up the bitbucket home directory in a future release.

Beginning with Bitbucket

This puppet module will automatically download the Bitbucket tar.gz from Atlassian and extracts it into /opt/bitbucket/atlassian-bitbucket-$version. The default Bitbucket home is /home/bitbucket.
Basic examples
1
2
3
class { 'bitbucket':
  javahome    => '/opt/java',
}
1
2
3
4
5
6
class { 'bitbucket':
  version        => '4.6.0',
  javahome       => '/opt/java',
  dburl          => 'jdbc:postgresql://bitbucket.example.com:5433/bitbucket',
  dbpassword     => $bitbucketpass,
}
Schedule a weekly git garbage collect for all repositories.
1
class { 'bitbucket::gc': }
Enable external facts for bitbucket version.
1
class { 'bitbucket::facts': }
Enable a bitbucket backup
1
2
3
4
5
6
7
8
9
10
class { 'bitbucket':
  backup_ensure          => present,
  backupclient_version    => '3.2.0',
  backup_home            => '/opt/bitbucket-backup',
  backupuser             => 'admin',
  backuppass             => 'password',
  backup_keep_age        => '3d',
  backup_schedule_hour   => '5',
  backup_schedule_minute => '0',
}
Upgrades
Upgrades to Bitbucket
Bitbucket can be upgraded by incrementing this version number. This will STOP the running instance of Bitbucket and attempt to upgrade. You should take caution when doing large version upgrades. Always backup your database and your home directory. The bitbucket::facts class is required for upgrades.
1
2
3
4
5
class { 'bitbucket':
  javahome => '/opt/java',
  version  => '4.6.0',
}
class { 'bitbucket::facts': }
If the bitbucket service is managed outside of puppet the stop_bitbucket paramater can be used to shut down bitbucket.
1
2
3
4
5
6
class { 'bitbucket':
  javahome   => '/opt/java',
  version    => '4.6.0',
  stop_bitbucket => 'crm resource stop bitbucket && sleep 15',
}
class { 'bitbucket::facts': }

Usage

This module also allows for direct customization of the JVM, following Atlassian's recommendations
This is especially useful for setting properties such as HTTP/https proxy settings. Support has also been added for reverse proxying bitbucket via Apache or nginx.

A more complex example

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
class { 'bitbucket':
  version        => '4.6.0',
  installdir     => '/opt/atlassian/atlassian-bitbucket',
  homedir        => '/opt/atlassian/application-data/bitbucket-home',
  javahome       => '/opt/java',
  download_url    => 'http://example.co.za/pub/software/development-tools/atlassian/',
  dburl          => 'jdbc:postgresql://dbvip.example.co.za:5433/bitbucket',
  dbpassword     => $bitbucketpass,
  service_manage => false,
  jvm_xms        => '1G',
  jvm_xmx        => '4G',
  java_opts      => '-Dhttp.proxyHost=proxy.example.co.za -Dhttp.proxyPort=8080 -Dhttps.proxyHost=proxy.example.co.za -Dhttps.proxyPort=8080 -Dhttp.nonProxyHosts=\"localhost|127.0.0.1|172.*.*.*|10.*.*.*|*.example.co.za\"',
  proxy          => {
    scheme       => 'https',
    proxyName    => 'bitbucket.example.co.za',
    proxyPort    => '443',
  },
  tomcat_port    => '7991'
}
class { 'bitbucket::facts': }
class { 'bitbucket::gc': }

A Hiera example

This example is used in production for 500 users in an traditional enterprise environment. Your mileage may vary. The dbpassword can be stored using eyaml hiera extension.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
# Bitbucket configuration
bitbucket::version:        '4.6.0'
bitbucket::installdir:     '/opt/atlassian/atlassian-bitbucket'
bitbucket::homedir:        '/opt/atlassian/application-data/bitbucket-home'
bitbucket::javahome:       '/opt/java'
bitbucket::dburl:          'jdbc:postgresql://dbvip.example.co.za:5433/bitbucket'
bitbucket::service_manage: false
bitbucket::download_url:    'http://example.co.za/pub/software/development-tools/atlassian'
bitbucket::jvm_xms:        '1G'
bitbucket::jvm_xmx:        '4G'
bitbucket::java_opts: >
  -XX:+UseLargePages
  -Dhttp.proxyHost=proxy.example.co.za
  -Dhttp.proxyPort=8080
  -Dhttps.proxyHost=proxy.example.co.za
  -Dhttps.proxyPort=8080
  -Dhttp.nonProxyHosts=localhost\|127.0.0.1\|172.*.*.*\|10.*.*.*\|*.example.co.za
bitbucket::env:
  - 'http_proxy=proxy.example.co.za:8080'
  - 'https_proxy=proxy.example.co.za:8080'
bitbucket::proxy:
  scheme:     'https'
  proxyName:  'bitbucket.example.co.za'
  proxyPort:  '443'
bitbucket::bitbucket_stop: '/usr/sbin crm resource stop bitbucket'

Reference

Classes

Public Classes

  • bitbucket: Main class, manages the installation and configuration of Bitbucket.
  • bitbucket::facts: Enable external facts for running instance of Bitbucket. This class is required to handle upgrades of Bitbucket. As it is an external fact, we chose not to enable it by default.
  • bitbucket::gc: Schedule a weekly git garbage collect for all repositories
  • bitbucket::backup: Schedule a backup of bitbucket

Private Classes

  • bitbucket::install: Installs Bitbucket binaries
  • bitbucket::config: Modifies Bitbucket/tomcat configuration files
  • bitbucket::service: Manage the Bitbucket service.

Parameters

Bitbucket parameters

javahome
Specify the java home directory. No assumptions are made re the location of java and therefore this option is required. Default: undef
version
Specifies the version of Bitbucket to install, defaults to latest available at time of module upload to the forge. It is recommended to pin the version number to avoid unnecessary upgrades of Bitbucket
format
The format of the file bitbucket will be installed from. Default: 'tar.gz'
installdir
The installation directory of the bitbucket binaries. Default: '/opt/bitbucket'
homedir
The home directory of bitbucket. Configuration files are stored here. Default: '/home/bitbucket'
manage_usr_grp
Whether or not this module will manage the bitbucket user and group associated with the install. You must either allow the module to manage both aspects or handle both outside the module. Default: true
user
The user that bitbucket should run as, as well as the ownership of bitbucket related files. Default: 'atlbitbucket'
group
The group that bitbucket files should be owned by. Default: 'atlbitbucket'
uid
Specify a uid of the bitbucket user. Default: undef
gid
Specify a gid of the bitbucket user: Default: undef
context_path
Specify context path, defaults to ''. If modified, Once Bitbucket has started, go to the administration area and click Server Settings (under 'Settings'). Append the new context path to your base URL.
tomcat_port
Specify the port that you wish to run tomcat under, defaults to 7990

database parameters

dbuser
The name of the database user that bitbucket should use. Default: 'bitbucket'
dbpassword
The database password for the database user. Default: 'password'
dburl
The uri to the bitbucket database server. Default: 'jdbc:postgresql://localhost:5432/bitbucket'
dbdriver
The driver to use to connect to the database. Default: 'org.postgresql.Driver'

JVM Java parameters

jvm_xms
Default: '256m'
jvm_xmx
Default: '1024m'
jvm_optional
Default: '-XX:-HeapDumpOnOutOfMemoryError'
jvm_support_recommended_args
Default: ''
java_opts
Default: ''

Tomcat parameters

proxy
Reverse https proxy configuration. See examples for more detail. Default: {}

Miscellaneous parameters

download_url
Where to download the bitbucket binaries from. Default: 'https://downloads.atlassian.com/software/stash/downloads'
checksum
The md5 checksum of the archive file. Only supported with deploy_module => archive. Defaults to 'undef'
service_manage
Should puppet manage this service? Default: true
$service_ensure
Manage the bitbucket service, defaults to 'running'
$service_enable
Defaults to 'true'
$stop_bitbucket
If the bitbucket service is managed outside of puppet the stop_bitbucket paramater can be used to shut down bitbucket for upgrades. Defaults to 'service bitbucket stop && sleep 15'
deploy_module
Module to use for installed bitbucket archive fille. Supports puppet-archive and nanliu-staging. Defaults to 'archive'. Archive supports md5 hash checking, Staging supports s3 buckets.
config_properties
Extra configuration options for bitbucket (bitbucket-config.properties). See https://confluence.atlassian.com/display/STASH/Bitbucket+config+properties for available options. Must be a hash, Default: {}
umask
Specify the umask bitbucket should run with. Defaults to undef, in which case the user account's default umask is left untouched.

Backup parameters

manage_backup
Whether to manage installation of backup client or not. Defaults to true.
backup_ensure
Enable or disable the backup cron job. Defaults to present.
backupclient_version
The version of the backup client to install. Defaults to '3.2.0'
backup_home
Home directory to use for backups. Backups are created here under /archive. Defaults to '/opt/bitbucket-backup'.
backupuser
The username to use to initiate the bitbucket backup. Defaults to 'admin'
backuppass
The password to use to initiate the bitbucket backup. Defaults to 'password'
backup_keep_age
How long to keep the backup archives for. You can choose seconds, minutes, hours, days, or weeks by specifying the first letter of any of those words (e.g., ‘1w’). Specifying 0 will remove all files.
backup_schedule_hour
Hour schedule for when to perform backup. Defaults to '5'.
backup_schedule_minute
Minute schedule for when to perform backup. Defaults to '0'.

Limitations

  • Puppet 3.4+
  • Puppet Enterprise
The puppetlabs repositories can be found at: http://yum.puppetlabs.com/ and http://apt.puppetlabs.com/
  • RedHat / CentOS 5/6/7
  • Ubuntu 12.04 / 14.04
  • Debian 7
We plan to support other Linux distributions and possibly Windows in the near future.

Development

Please feel free to raise any issues here for bug fixes. We also welcome feature requests. Feel free to make a pull request for anything and we make the effort to review and merge. We prefer with tests if possible.

Testing - How to test the Bitbucket module

Using puppetlabs_spec_helper. Simply run:
1
bundle install && bundle exec rake spec
to get results.
1
2
3
4
5
6
/usr/bin/ruby1.9.1 -S rspec spec/classes/bitbucket_config_spec.rb spec/classes/bitbucket_facts_spec.rb spec/classes/bitbucket_install_spec.rb spec/classes/bitbucket_service_spec.rb spec/classes/bitbucket_upgrade_spec.rb --color
ldapname is deprecated and will be removed in a future version
.......................

Finished in 2.02 seconds
23 examples, 0 failures
run (Additional yak shaving may be required):
1
2
3
BEAKER_set=ubuntu-server-12042-x64 bundle exec rake beaker
BEAKER_set==debian-73-x64 bundle exec rake beaker
BEAKER_set==centos-64-x64 bundle exec rake beaker

Contributors

  • Jaco Van Tonder
  • Merritt Krakowitzer merritt@krakowitzer.com
  • Sebastian Cole
  • Geoff Williams
  • Bruce Morrison
  • Daniel Duwe
  • Brian Carpio
  • Frank Kleine
  • Ben Roberts

Wednesday, 22 February 2017

Configure SSL between the IBM HTTP Server Administration Server and the deployment manager

Configure Secure Sockets Layer (SSL) between the deployment manager for WebSphere® Application Server and the IBM® HTTP Server administration server, which is called adminctl.

About this task

The Application Server has new SSL management functions that need to be managed properly in order for IBM HTTP Server to connect with an SSL request. In earlier releases, SSL connections used default dummy certificates that were exchanged between IBM HTTP Server and the Application Server. In WebSphere Application Server, you must configure the Application Server to accept a self-signed certificate from IBM HTTP Server so SSL connections are accepted and transactions are completed.
If the Application Server and the IBM HTTP Server administration server are not configured correctly, the Application Server shows any errors that are received in the log file for the deployment manager. In situations where the IBM HTTP Server administration server is attempting to connect through SSL and the Application Server is not configured, you might receive an error that is similar to the following message:
-CWPKI0022E: SSL HANDSHAKE FAILURE:  A signer with
SubjectDN "CN=localhost" was sent from target host:port "null:null".
The signer may need to be added to local trust store "c:/619/app2/profiles/Dmgr01/config/cells/rjrCell02/trust.p12" 
located in SSL configuration alias "CellDefaultSSLSettings"
 loaded from SSL configuration file "security.xml".  
The extended error message from the SSL handshake
 exception is: "No trusted certificate found".

-IOException javax.net.ssl.SSLHandshakeException: 
com.ibm.jsse2.util.h: No trusted certificate found

Procedure

  1. Obtain a server certificate. You can generate a new self-signed certificate or use the existing certificate from the IBM HTTP Server Web server plugin.
    • Use the existing self-signed certificate from the IBM HTTP Server Web server plugin.
    • Create a CMS key database file and a self-signed server certificate. Use the iKeyman utility for distributed operating systems and the gskkyman tool for z/OS® operating systems. This step and later steps will assume that you are using the iKeyman utility.
      • [AIX Solaris HP-UX Linux Windows] Use the IBM HTTP Server iKeyman utility graphical user interface or command line to create a CMS key database file and a self-signed server certificate.
        Use the iKeyman utility to create a self-signed certificate for the IBM HTTP Server Administration Server and save the certificate as /conf/admin.kdb.
        Best practice
        Make note of the password and select Stash password to a file.
        The following fields are required for the certificate:
        Label
        adminselfSigned
        Common Name
        fully_qualified_host_name
      • [z/OS] IBM HTTP Server uses the z/OS gskkyman tool for key management to create a CMS key database file, public and private key pairs, and self-signed certificates. Alternatively, you can create a SAF keyring in place of a CMS key database file.
        • For information on gskkyman, see Key management using the native z/OS key database.
        • For information on creating SAF keyrings, see Authenticating with SAF on IBM HTTP Server and SSL keyfile directive.
  2. Extract the certificate to a file using iKeyman utility.
    1. Select the certificate that you created in Step 1. For example, adminselfSigned.
    2. Click Extract Certificate. The recommended file name for extraction is C:\Program Files\IBM\HTTPServer\conf\cert.arm.
      Avoid trouble
      Do not change the data type.
  3. Modify the Administration Server configuration File, which is named admin.conf.
    1. Configure the file to load the IBM SSL module. Uncomment the following line:
      LoadModule ibm_ssl_module     modules/mod_ibm_ssl.so
    2. Enable SSL and define a key file to use. Uncomment the following lines to enable SSL and define a key file to use:
      SSLEnable
      SSLServerCert default
      Keyfile "C:/Program Files/IBM/HTTPServer5/conf/admin.kdb"
      Avoid trouble
      Be aware of the following:
      • The key file directive must match the name and location of a valid key file that is installed on your system.
      • You must have IBM SSL support installed for this to work.
      • The "default" in SSLServerCert is the label, or name, of the self-signed certificate that is created when the plugin-key.kdb file was created.
      • The previous example uses SSLServerCert because the default self-signed certificate in the plugin-key.kdb is not flagged as the default certificate.
  4. Start the administration server for IBM HTTP Server. Verify that the log file does not contain GSKIT errors.
  5. Configure WebSphere Application Server.
    1. Log into the Administrative Console for the Application Server and start the deployment manager.
    2. Select Security > SSL certificate and key management.
    3. Select Manage endpoint security configurations. You are directed to a list of inbound and outbound endpoints.
    4. Select the outbound cell (cellDefaultSSLSettings,null). Select outbound cells because, in this setup, the Administration Console for the Application Server is the client, and the IBM HTTP Server Administration Server is the server.
      Avoid trouble
      This setup is the opposite configuration from an SSL setup with the IBM HTTP Server plugin and the Application Server.
    5. In the Related Items section, click Key stores and certificates.
    6. Click CellDefaultTrustStore.
    7. In the Additional Properties section, click Signer Certificates.
    8. FTP the certificate file to the Application Server. Do not change the data type.
    9. In the collection panel for Signer Certificates, click Add. Enter the following information in the fields.
      Table 1. Signer Certificate information
      NameValue
      AliasadminselfSigned
      File namefile_name
      For example, enter the following:
      c:\program files\ibm\httpserver\conf\cert.arm
    10. Save the configuration changes to the administrative console.
    11. Stop the deployment manager.
    12. Start the deployment manager.

Results

The IBM HTTP Server administration server and Application Server are now configured to use SSL transactions.

Saturday, 14 May 2016

Create schedulers in websphere.

You can create multiple schedulers within a single server, cluster, node or cell. Each configured scheduler is an independent task scheduling engine that has a unique Java Naming and Directory Interface (JNDI) name, persistent storage device and daemon.

Procedure

  1. Configure schedulers.
  2. Create the database for schedulers(same as our data source creation)

1) Set scheduler in WebSphere console (you'll need database connection set as well, because scheduler automaticly creates few tables for managing tasks)

Go to resources -> schedulers -> new

Set JNDI name by which you'll call Scheduler from your application (in this example, it will be sched/testScheduler), and ofcourse choose database conneciton.

Poll interval - i'm not sure what is purpose of it because i set interval for tasks in application, but anyway i set it to 30min.

Table prefix - i guess choose whatever you prefer

Now after you created new Scheduler, select it in table (resources -> schedulers) and click 'Create tables'.

Results

A scheduler is now configured and ready to use for newly installed applications. If the scheduler JNDI name is not yet visible to your application, restarting the application or restarting application server will allow the scheduler to be seen.
When schedulers are created for the first time, the poll daemon will not automatically start and must be started manually and will only start automatically the next time the server is started. To start the poll daemon manually, refer to the scheduler daemons topic.
Attention: Changes to existing scheduler configurations will not take affect until after the application server is restarted.


Friday, 19 February 2016

WebSphere portal content will share soon..


How to enable probe in DataPower

Login to DP console  >> > control panel

Click on required services -à Click on Services name

Click on Show Probe button à click on Enable probe.