CentOS LXRng - Postgres Howto

Submitted by admin on

If you are doing any Kernel development, using any of the available APIs or hacking away at it for various reasons, LXR can be a developers best friend. And lets face it, you (or your developers) saving time is money/time well spent in the eyes of management. LXR has allowed me and others to determine changes in the Kernel, further understanding of subsystems and allowed us to port patches to newer revisions. So a few times we have attempted without all that much success - our own LXR indexer.

If there has been one thing that has really frustrated me with LXR is the lack of complete documentation for LXR/LXRng. I have attempted this for unsuccessfully Fedora and successfully for Ubuntu in the past. Recently I reattempted this issue and managed to get it running on CentOS 5.9. Here are the steps that I used:

In case you didn't know what LXR is:

LXR is a general purpose source code indexer and cross-referencer that provides web-based browsing of source code with links to the definition and usage of any identifier. It supports a wide choice of languages. LXR used to stand for "Linux Cross-Referencer" but it has proved generic enough to be able to cross reference any project beyond the Linux kernel. It is based on stock web technology (meaning its output is regular HTML), so the display client may be freely chosen from the available web browsers. It was a design choice not to use Java or any other client-side scripting language to minimise dependencies and system requirements. - taken from the LXR projects website

This tutorial assumes that you have some sort of understanding about Linux and have some sort of computing competency and while no guarentees can be given - I may have missed a step or two recompiling this howto (so feel free to fire me an email or leave a comment below) .

Initial System Setup

Start by plugging in whatever media you need to setup

Then configure the system as Gnome Desktop and configure now. Press next.

Now select the categories you would like to install - under development choose Development Tools and Development Libraries. Also select under Servers, Web Server and Postgres SQL Server.

Continue the installation until it has finished, and at the firewall prompt choose to allow HTTP & SSH (optional). I disabled SeLinux for the purposes of this document.

Add RHEL, Xapian, RPMForge Repos

By default, the CentOS distribution needs a larger base in which to install packages from. The following steps will allow you to enable the above package repositories

  1. wget http://dl.fedoraproject.org/pub/epel/5/i386/epel-release-5-4.noarch.rpm
  2. sudo rpm -Uvh  epel-release-5*.rpm

Now install the RPM Forge repository

  1. wget http://packages.sw.be/rpmforge-release/rpmforge-release-0.5.2-2.el5.rf.x86_64.rpm
  2. sudo rpm --import http://apt.sw.be/RPM-GPG-KEY.dag.txt
  3. sudo rpm -ivh rpmforge-release-0.5.2-2.el5.rf.x86_64.rpm  

Now install the Xapian repository (this is for the LXR searching code)

  1. sudo rpm -ivh http://rpm.eprints.org/rpm-eprints-org-key-1-1.noarch.rpm
  2. sudo rpm -ivh http://rpm.eprints.org/xapian/6/noarch/rpm-eprints-org-xapian-6-1.noarch.rpm

Now lets update the system

sudo yum update

Install Prerequisities

Now lets install the following:

yum install git inkscape postgresql postgresql-server ctags perl-CGI-Ajax perl-CGI-Simple perl-DBD-Pg perl-HTML-Parser perl-Template-Toolkit perl-Term-ProgressBar perl-Term-Size perl-TermReadKey perl-Devel-Size gcc gcc-c++  perl-ExtUtils-MakeMaker perl-CPAN perl-YAML libpng libpng-devel texinfo perl-Digest-SHA1 wget httpd make icoutils xapian-core gifsicle xapian-bindings-{php,python,tcl8,perl,ruby} ncftp

Using Perl's CPAN shell, configure it and install the following:

  1. sudo perl -MCPAN -e shell
  2.  
  3. /usr/lib/perl5/5.8.8/CPAN/Config.pm initialized.
  4.  
  5.  
  6. CPAN is the world-wide archive of perl resources. It consists of about
  7. 100 sites that all replicate the same contents all around the globe.
  8. Many countries have at least one CPAN site already. The resources
  9. found on CPAN are easily accessible with the CPAN.pm module. If you
  10. want to use CPAN.pm, you have to configure it properly.
  11.  
  12. If you do not want to enter a dialog now, you can answer 'no' to this
  13. question and I'll try to autoconfigure. (Note: you can revisit this
  14. dialog anytime later by typing 'o conf init' at the cpan prompt.)
  15.  
  16. Are you ready for manual configuration? [yes] yes
  17.  
  18.  
  19. The following questions are intended to help you with the
  20. configuration. The CPAN module needs a directory of its own to cache
  21. important index files and maybe keep a temporary mirror of CPAN files.
  22. This may be a site-wide directory or a personal directory.
  23.  
  24.  
  25.  
  26. First of all, I'd like to create this directory. Where?
  27.  
  28. CPAN build and cache directory? [/home/rbrash/.cpan]
  29.  
  30.  
  31. If you want, I can keep the source files after a build in the cpan
  32. home directory. If you choose so then future builds will take the
  33. files from there. If you don't want to keep them, answer 0 to the
  34. next question.
  35.  
  36.  
  37.  
  38. How big should the disk cache be for keeping the build directories
  39. with all the intermediate files?
  40.  
  41. Cache size for build directory (in MB)? [10]
  42.  
  43.  
  44. By default, each time the CPAN module is started, cache scanning
  45. is performed to keep the cache size in sync. To prevent from this,
  46. disable the cache scanning with 'never'.
  47.  
  48. Perform cache scanning (atstart or never)? [atstart]
  49.  
  50.  
  51. To considerably speed up the initial CPAN shell startup, it is
  52. possible to use Storable to create a cache of metadata. If Storable
  53. is not available, the normal index mechanism will be used.
  54.  
  55. Cache metadata (yes/no)? [yes]
  56.  
  57.  
  58. The next option deals with the charset your terminal supports. In
  59. general CPAN is English speaking territory, thus the charset does not
  60. matter much, but some of the aliens out there who upload their
  61. software to CPAN bear names that are outside the ASCII range. If your
  62. terminal supports UTF-8, you say no to the next question, if it
  63. supports ISO-8859-1 (also known as LATIN1) then you say yes, and if it
  64. supports neither nor, your answer does not matter, you will not be
  65. able to read the names of some authors anyway. If you answer no, names
  66. will be output in UTF-8.
  67.  
  68. Your terminal expects ISO-8859-1 (yes/no)? [yes]
  69.  
  70.  
  71. If you have one of the readline packages (Term::ReadLine::Perl,
  72. Term::ReadLine::Gnu, possibly others) installed, the interactive CPAN
  73. shell will have history support. The next two questions deal with the
  74. filename of the history file and with its size. If you do not want to
  75. set this variable, please hit SPACE RETURN to the following question.
  76.  
  77. File to save your history? [/home/rbrash/.cpan/histfile]
  78. Number of lines to save? [100]
  79.  
  80.  
  81. The CPAN module can detect when a module that which you are trying to
  82. build depends on prerequisites. If this happens, it can build the
  83. prerequisites for you automatically ('follow'), ask you for
  84. confirmation ('ask'), or just ignore them ('ignore'). Please set your
  85. policy to one of the three values.
  86.  
  87. Policy on building prerequisites (follow, ask or ignore)? [ask]
  88.  
  89.  
  90. The CPAN module will need a few external programs to work properly.
  91. Please correct me, if I guess the wrong path for a program. Don't
  92. panic if you do not have some of them, just press ENTER for those. To
  93. disable the use of a download program, you can type a space followed
  94. by ENTER.
  95.  
  96. Where is your gzip program? [/usr/bin/gzip]
  97. Where is your tar program? [/bin/tar]
  98. Where is your unzip program? [/usr/bin/unzip]
  99. Where is your make program? [/usr/bin/make]
  100. Where is your links program? [/usr/bin/links]
  101. Where is your wget program? [/usr/bin/wget]
  102. Warning: ncftpget not found in PATH
  103. Where is your ncftpget program? []
  104. Where is your ncftp program? [/usr/bin/ncftp]
  105. Where is your ftp program? [/usr/bin/ftp]
  106. Where is your gpg program? [/usr/bin/gpg]
  107. What is your favorite pager program? [/usr/bin/less]
  108. What is your favorite shell? [/bin/bash]
  109.  
  110.  
  111. Every Makefile.PL is run by perl in a separate process. Likewise we
  112. run 'make' and 'make install' in processes. If you have any
  113. parameters (e.g. PREFIX, LIB, UNINST or the like) you want to pass
  114. to the calls, please specify them here.
  115.  
  116. If you don't understand this question, just press ENTER.
  117.  
  118. Parameters for the 'perl Makefile.PL' command?
  119. Typical frequently used settings:
  120.  
  121.     PREFIX=~/perl       non-root users (please see manual for more hints)
  122.  
  123. Your choice:  []
  124. Parameters for the 'make' command?
  125. Typical frequently used setting:
  126.  
  127.     -j3              dual processor system
  128.  
  129. Your choice:  []
  130. Parameters for the 'make install' command?
  131. Typical frequently used setting:
  132.  
  133.     UNINST=1         to always uninstall potentially conflicting files
  134.  
  135. Your choice:  []
  136.  
  137.  
  138. Sometimes you may wish to leave the processes run by CPAN alone
  139. without caring about them. As sometimes the Makefile.PL contains
  140. question you're expected to answer, you can set a timer that will
  141. kill a 'perl Makefile.PL' process after the specified time in seconds.
  142.  
  143. If you set this value to 0, these processes will wait forever. This is
  144. the default and recommended setting.
  145.  
  146. Timeout for inactivity during Makefile.PL? [0]
  147.  
  148.  
  149. If you're accessing the net via proxies, you can specify them in the
  150. CPAN configuration or via environment variables. The variable in
  151. the $CPAN::Config takes precedence.
  152.  
  153. Your ftp_proxy?  
  154. Your http_proxy?  
  155. Your no_proxy?  
  156. You have no /home/rbrash/.cpan/sources/MIRRORED.BY
  157.   I'm trying to fetch one
  158. CPAN: LWP::UserAgent loaded ok
  159. Fetching with LWP:
  160.   ftp://ftp.perl.org/pub/CPAN/MIRRORED.BY
  161.  
  162.  
  163. Now we need to know where your favorite CPAN sites are located. Push
  164. a few sites onto the array (just in case the first on the array won't
  165. work). If you are mirroring CPAN to your local workstation, specify a
  166. file: URL.
  167.  
  168. First, pick a nearby continent and country (you can pick several of
  169. each, separated by spaces, or none if you just want to keep your
  170. existing selections). Then, you will be presented with a list of URLs
  171. of CPAN mirrors in the countries you selected, along with previously
  172. selected URLs. Select some of those URLs, or just keep the old list.
  173. Finally, you will be prompted for any extra URLs -- file:, ftp:, or
  174. http: -- that host a CPAN mirror.
  175.  
  176. (1) Africa
  177. (2) Asia
  178. (3) Europe
  179. (4) North America
  180. (5) Oceania
  181. (6) South America
  182. Select your continent (or several nearby continents) [] 4
  183. Sorry! since you don't have any existing picks, you must make a
  184. geographic selection.
  185.  
  186. (1) Canada
  187. (2) Costa Rica
  188. (3) Mexico
  189. (4) United States
  190. Select your country (or several nearby countries) [] 1
  191. Sorry! since you don't have any existing picks, you must make a
  192. geographic selection.
  193.  
  194. (1) ftp://CPAN.mirror.rafal.ca/pub/CPAN/
  195. (2) ftp://cpan.sunsite.ualberta.ca/pub/CPAN/
  196. (3) ftp://ftp.nrc.ca/pub/CPAN/
  197. (4) ftp://mirror.arcticnetwork.ca/pub/CPAN/
  198. (5) ftp://mirror.csclub.uwaterloo.ca/CPAN/
  199. (6) ftp://mirror.its.dal.ca/cpan/
  200. (7) ftp://mirror.vexxhost.com/cpan/
  201. (8) http://cpan.perlnow.com/
  202. (9) http://httpupdate25.cpanel.net/CPAN/
  203. (10) http://mirrors.gossamer-threads.com/CPAN/
  204. Select as many URLs as you like (by number),
  205. put them on one line, separated by blanks, e.g. '1 4 5' [] 1 2 3 4 5 6 7 8 9 10
  206.  
  207. Enter another URL or RETURN to quit: []
  208. New set of picks:
  209.   ftp://CPAN.mirror.rafal.ca/pub/CPAN/
  210.   ftp://cpan.sunsite.ualberta.ca/pub/CPAN/
  211.   ftp://ftp.nrc.ca/pub/CPAN/
  212.   ftp://mirror.arcticnetwork.ca/pub/CPAN/
  213.   ftp://mirror.csclub.uwaterloo.ca/CPAN/
  214.   ftp://mirror.its.dal.ca/cpan/
  215.   ftp://mirror.vexxhost.com/cpan/
  216.   http://cpan.perlnow.com/
  217.   http://httpupdate25.cpanel.net/CPAN/
  218.   http://mirrors.gossamer-threads.com/CPAN/
  219.  
  220.  
  221. commit: wrote /usr/lib/perl5/5.8.8/CPAN/Config.pm
  222. Terminal does not support AddHistory.
  223.  
  224. cpan shell -- CPAN exploration and modules installation (v1.7602)
  225. ReadLine support available (try 'install Bundle::CPAN')

Now the modules

  1. sudo perl -MCPAN -e 'install Search::Xapian'
  2. sudo perl -MCPAN -e 'install Class::Accessor'
  3. sudo perl -MCPAN -e 'install Class::MethodMaker'
  4. sudo perl -MCPAN -e 'install Devel::Size'
  5. sudo perl -MCPAN -e 'install Digest::SHA'
  6. sudo perl -MCPAN -e 'install HTML::Parser'
  7. sudo perl -MCPAN -e 'install IO::Stringy'
  8. sudo perl -MCPAN -e 'install PerlIO::gzip'
  9. sudo perl -MCPAN -e 'install Template'
  10. sudo perl -MCPAN -e 'install HTML::Entities'

Install and Configure LXR

  1. cd /opt
  2. sudo git clone git://lxr.linux.no/git/lxrng.git
  3. cd lxrng
  4. cp lxrng.conf-dist lxrng.conf
  5. cp apache2-site.conf-dist-cgi apache2-site.conf

Now change your user to root:

  1. su -
  2. chkconfig httpd on
  3. chkconfig postgresql on
  4. service postgresql start
  5. /etc/init.d/httpd start
  6. exit

Now change your user to the postgres user

  1. su -
  2. su - postgres
  3. createdb lxrng
  4.  
  5. createuser www-data
  6. Shall the new role be a superuser? (y/n) n
  7. Shall the new role be allowed to create databases? (y/n) n
  8. Shall the new role be allowed to create more new roles? (y/n) n
  9. CREATE ROLE
  10.  
  11. createuser apache
  12. Shall the new role be a superuser? (y/n) n
  13. Shall the new role be allowed to create databases? (y/n) n
  14. Shall the new role be allowed to create more new roles? (y/n) n
  15. CREATE ROLE
  16.  
  17. createuser lxr
  18. Shall the new role be a superuser? (y/n) n
  19. Shall the new role be allowed to create databases? (y/n) n
  20. Shall the new role be allowed to create more new roles? (y/n) n
  21. CREATE ROLE
  22.  
  23. createuser lxradmin
  24. Shall the new role be a superuser? (y/n) n
  25. Shall the new role be allowed to create databases? (y/n) y
  26. Shall the new role be allowed to create more new roles? (y/n) n
  27. CREATE ROLE
  28.  
  29. cd /opt/lxrng/

Create and change the permissions for the following directories

  1. sudo mkdir /var/lib/lxrng/text-db -p
  2. sudo mkdir /var/lib/lxrng/cache
  3. sudo chmod a+rw /var/lib/lxrng
  4. sudo chmod a+rw /var/lib/lxrng/cache/

We are getting close to finishing up here, grab a coffee after starting the download:

  1. sudo make -C /opt/lxrng/webroot/.static/gfx
  2. sudo mkdir -p /var/lib/lxrng/repos/
  3. sudo cd /var/lib/lxrng/repos/
  4. sudo hit clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git
  5. sudo cd linux-2.6/
  6. sudo git tag

Edit the configuration file using sed

  1. cd /opt/lxrng/
  2. sudo sed -i 's:@@LXRURL@@::' apache2-site.conf
  3. sudo sed -i 's:@@LXRROOT@@:/opt/lxrng:' apache2-site.conf

Uncomment the following line in: apache2-site.conf

  1. # For LXRng installed directly in the web site root, use
  2. ScriptAlias /            "/opt/lxrng/webroot/lxr.cgi/"
  3. # otherwise use (no trailing slash):
  4. #   ScriptAlias /  "/opt/lxrng/webroot/lxr.cgi"
  5. ...

Edit lxrng.conf using vi lxrng.conf

  1. ...
  2.         'base_url'    =>'http://localhost/',
  3. ...
  4.         'ver_default' =>; 'v2.6.34',
  5. ...

Now prepare a symlink

sudo ln -s /opt/lxrng/apache2-site.conf /etc/httpd/conf.d/.

Setup the postgres database and start the lxr referencing

  1. su -
  2. su - postgres
  3.  ./lxr-db-admin linux --init
  4. ./lxr-genxref linux
  5. exit

Restart apache and goto localhost in your browser of choice

sudo /etc/init.d/httpd restart

And now your LXR installation should be up and running locally :)

References

http://xapian.org/download
http://www.rackspace.com/knowledge_center/article/installing-rhel-epel-repo-on-centos-5x-or-6x
http://perl.about.com/od/packagesmodules/qt/perlcpan.htm
https://github.com/yasuaki/lxrng/blob/master/INSTALL
http://mohammadthalif.wordpress.com/2010/07/24/how-to-setup-lxr-–-step-by-step-guide-3/
http://www.sungju.org/?p=434
http://wiki.centos.org/AdditionalResources/Repositories/RPMForge

AttachmentSize
LXR_setup1.PNG139.61 KB
LXR_setup2.PNG153.73 KB
LXR_setup3.PNG141.44 KB
LXR_setup4.PNG73.56 KB

Add new comment

More information about text formats

Filtered HTML

  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
By submitting this form, you accept the Mollom privacy policy.