We are thrilled to announce that we have been empanelled with HARTRON (HARYANA STATE ELECTRONICS DEVELOPMENT CORPORATION LIMITED) as Strategic/Channel partners for Website Development and Implementation.

We would now be able to bid for large e-Governance projects which we have been seriously missing in our portfolio for sometime now.

Thanks,

Hariprasad.

Srijan is looking for the following a CSS designer with the following profile:

• create pixel-perfect CSS designs from PSD files provided
• cross-browser compatible CSS
• experience working with atleast one CSS framework, such as 960 GS, Blueprint, Yahoo! UI Library, YAML, others
• experience in jQuery and other Javascript frameworks is preferred

The selected candidate would be working on Drupal and TYPO3 CMS in designing and deploying international quality websites. We prefer women candidates (our male:female ratio at office has become extremely skewed, setting a bad example for a company like us).

To apply, please write at jobs[at]srijan[dot]in, with the subject marked “CSS Designer”

Srijan is looking for a “Head of Marketing“. See the job description here (PDF). A summary is given below:

• an MBA from tier-1 or tier-2 MBA institutes in India
• experience  in corporate and institutional sales, in selling software or software services (such as corporate websites, portals, e-commerce websites, web-based applications)
• experience in new client acquisition through direct sales, and reference sales made through current customer
• experience in generating international leads specifically in USA and Europe
• experience gathering, documenting requirements, and writing proposals for small to medium sized projects ($10,000 to $100,000)
• can negotiate contracts for websites, portals, and web-based business applications
• can write technical and commercial proposals for such contracts
• identify industry verticals where existing solutions built by Srijan over the years could be re-used
• write SLAs for maintenance of these projects
• very well networked
• experience in using social networking tools such as LinkedIn, Xing and other business networking sites, to generate business leads
• uses Twitter (and other similar tools) for networking and web branding
• experience in lead generation and sales of CMS solutions based on proprietary or open source software
• familiar with Drupal, TYPO3, Joomla or other Content Management Systems
• manage getting publicity for Srijan in web and print media to position Srijan’s brand, using his/her personal media network
• experience in writing business plans for products
• create a web marketing strategy for selling SaaS products
• highly self-motivated
• good value systems
• preferably an educational background in technology (Diploma/Bachelors in computers)
• owns a Blog and writes often on a wide-variety of topics, specifically technology and marketing
• experience in defining Google Adwords marketing; testing and redoing the strategy for maximising campaign effectiveness

Here comes our corporate presentation covering our important TYPO3 and Drupal projects and development processes.

Press Release

Contact:
Rahul Dewan
rahul[at]srijan.in
Srijan Technologies Pvt. Ltd.
304 Bakshi House
40-41 Nehru Place New Delhi 110019

Srijan Technologies among the world’s “Most Democratic Workplaces”

Srijan Technologies Pvt. Ltd. a New Delhi based web-development company has been selected as one of the world’s “Most Democratic Workplaces” by WorldBlu, an Austin, Texas (USA) based non-profit organization. WorldBlu specializes in issues relating to democracy in the workplace. Srijan has been selected to be part of WorldBlu’s 3rd annual list of 40 “Most Democratic Workplaces” released on April 14th, 2009.

Srijan (www.srijan.in) specializes in delivering high end, sophisticated websites to clients around the world. It uses open source technologies such as Drupal, TYPO3, Apache, PHP, MySQL and Linux to make high quality, reliable and engaging websites. Srijan’s philosophy is not only to use open source technologies but to have an “open” and democratic workplace.

At Srijan important decisions relating to overall company strategy, organization, employee salaries and promotions are made from the bottom-up. This decision making happens in public meetings involving all employees at Srijan. Any employee is free to call, lead and participate in these public meetings. Decisions requiring detailed analysis happen in groups selected by the employees. After research and internal discussion these groups present their findings to all employees and the final decision is taken by consensus. Focus and goal orientation is high at Srijan because employees are aware of where their company is headed; what’s more, they have helped choose that direction. At Srijan, democracy does not imply chaos: Senior management still plays an important role because it brings experience and perspective to the table. The major difference from traditional top-down organizations is that senior management cannot ask people to do something simply because they “said so.” Employees as a whole need to be truly convinced by an idea, direction or initiative. Srijan believes its way of functioning is unique in India where participative management is followed more in theory than in actual practice.

An expanding set of case studies prove that more transparency and democracy is good for every organization. The writings of Ricardo Sembler (celebrated author of “Maverick!” and CEO of Semco) are a famous case in point. The democratic approach has many dividends: the Srijan workplace tends to be remarkably non-political, positive and task focused rather than personality focused. Moreover, Srijan employees consistently describe their organization as “fair.” Srijan practices “democracy at the workplace” not only because its the right thing to do, but it’s the profitable thing to do.

WorldBlu reports that this year, non-profit and for-profit organizations from the U.S., Canada, Mexico, England, India, the Netherlands, Switzerland, and Malaysia made the “Most Democratic Workplace” list from industries such as technology, health care, telecommunications, media, manufacturing, aerospace and retail. According to WorldBlu they represent nearly $12 billion (approx Rs. 63,000 crores) in combined annual sales. Srijan Technologies was the only company selected from India. These companies were selected on a basis of a rigorous, anonymous questionnaire filled by employees of the candidate organizations. All Srijan employees were eligible to fill this questionnaire and an overwhelming majority eventually did.

“Nothing has proven the need for a new model of business as definitively as the current economic crisis,” says WorldBlu (www.worldblu.com) Founder and President, Traci Fenton. “The call for transparent, accountable, and decentralized companies is greater than ever, and in the midst of all the economic turmoil these organizations are a bright spot and a reason for hope.”

Srijan’s inclusion in this top 40 global list is a great motivator to our employees and validation of our core values. Srijan is proud to be part of this list.

More information

• See Srijan’s story published at Worldblu
• Learn more about Srijan at www.srijan.in
• See the top 40 list at worldblu.com
• Worldblu List 2009 – Press Release

Srijan has joined Acquia’s partner network, as a Silver Partner.

Acquia Silver Partner

Associating with Acquia has been a high priority for us, as Srijan is going to be focussing extensively on Drupal in the coming months. There are already two large projects we are currently executing for a recently launched News and Features magazine in India, and another one for a broadcasting agency in Greenland. We are expecting more such large Drupal projects in the coming months, and Acquia’s tech support model for its partners would prove to be beneficial for us in executing of these projects.

It feels good to be associated with them, specifically as one of our former Drupal mentors works for Acquia now.

Find us at Acquia’s site: http://acquia.com/partners/showcase/srijan-technologies-pvt-ltd

Deepak Chopra, during a recent visit to New Delhi, gave an interview on CNBC-TV18. I loved what he said, and could not resist making a post about the interview. See the complete interview transcript.

An excerpts from the interview:

…we have been so programmed to think about me and mine. Now the fact is the world is showing us right now that there is no such thing as me and mine. The economic meltdown is global. There is no national solution to it. If there is going to be a solution it is going to have to be global. Terrorism is a global problem. If there is going to be a solution it is going to have to be a global solution. The ecosystem and the damage that we have inflicted on the environment, which is global warming is a global problem. So, in a sense we are being forced to recognise our interdependence. A few years ago this was a philosophy, now it is a fact.

Over the past year, we at Srijan Labs have been dabbling in a variety of areas. One key area of interest has been the possibilities afforded by GPS tracking, FOSS GIS tools, and location-based services centred around these. Also, of interest to us is how these services can be harnessed through mobile devices like cell-phones, and hand-held devices that are becoming increasingly ubiquitous.

With this in mind, I have put down some thoughts in a document available here as PDF. Nothing earthshaking, but just an idea of the areas which we wish to explore in the immediate future. Comments
are invited.

Download PDF article

As part of our community initiatives, we are sponsoring the Barcamp Kashmir 2009. It feels good to be part of a community event in Kashmir.

As part of it’s objectives of promoting open source adoption, and in specific proliferate Drupal in India, Srijan co-sponsored the Delhi Drupal meetup, at IIT Delhi on 01 March 2009.

We’re working on integrating Apache Solr with TYPO3 at the moment. We had started work sometime back, and recently during a project assignment from Sweden, we realised that Apache Solr would be a best fit in this case, as there is a lot of Meta content associated with the primary search content in this portal that we’re building.

We feel institutions working in the following areas can get immense benefit from implementing Solr in their IT systems as a search and filter tool:

• Newspapers and Journals
• Lawyers
• Libraries
• Stock Photo portals
• Stock photo studios
• Advertising agencies for their stock repository

To see Solr in action (even though this is on Drupal) please visit: http://www.indiaenvironmentportal.org.in/search/apachesolr_search/. A case study of the project is available at: http://blogs.srijan.in/2008/10/10/india-environment-portal-a-case-study/.

This is a first draft presentation we prepared which talks about Drupal theme advantages versus TYPO3 templating practices. We’re working on a more detailed version of this to cover code aspects as well. Stay tuned! See the presentation (PDF).

a lot of times one is confronted with a situation where the design team is asked to create and show ’something’ even before a project has been awarded. lot of companies conduct this exercise with multiple service providers who ‘work for free’, expecting a good project at the end of it.

i have always wondered about the logic behind this practice. monoliths do this because they have inherited this as a legacy. it serves as yet-another-process put in place to justify a decision (‘justify’ is the key word here ) and others are simply following the age old tradition of being part of the herd by going where the first sheep goes:)

please do not see this as ranting of a person involved with design. i may be using some not-so-soft words, metaphors and analogies but that is because i also need to clearly state my point of view on a topic which often becomes a bone of contention between a customer and a service provider. why i oppose this is not just because it harms me but also because it harms the customers the most!

now one may ask – how the hell does this adversely impact a customer?

before answering that question, let me clarify that i am specifically talking about the internet here though this logic can be extended to other mediums as well.

a customer is looking forward to a ‘solution’ – a communication solution, a business solution… the role and goals on the intervention are defined by the business proposition of the intervention. there is a certain need which has to addressed and certain targets that have to be met. but in no way this is just about form and colours!

a solution has to aesthetically pleasing, no doubt. if it is a trendsetter, its just great! but to set trends one needs a lot of courage and some serious decision making pertaining to design. and to do that one has to dig deeper into what forms and colours and tones mean – in the context of how viewers/readers perceive them, the socio-cultural connotation and so on. by simply stating that the ‘wow factor’ is missing does not help anyone.

just like business processes, products, service offerings, supply chains, strategy… design also evolves. one thought leads to another and slowly a ‘solution’ emerges. more than ‘wow’, it is about being wonderful.

and how does it become wonderful – by working hard!

and how does it work hard – not merely by looking good but by being logical and by serving the end users well!

now what that means is that your designer needs to understand your business damn well. he/she needs to know what you are upto and where you are heading. and most of all, he/she needs to be capable of understanding and digesting all this. this can be only deciphered by talking and discussing approaches with him/her and not by ogling at some design which follows some contemporary and popular design trend. your team, zapped by some latest booze ad, may fall in love with the proposed floral patterns in the page background. or on those nifty icons rendered in 3-d. not once will they question whether it goes well with the geometric pattern of your logo. impact of persuasive graphics will far exceed the non-persuasive elements of your communication. and as an insider, let me tell you, the designer of the speculative work will gun for this sentiment alone. moreover, a lot of the ‘worthwhile’ folks in design would refuse to participate. not because they are elitist but because they know that we are in the business of creating solutions and not pop art.

so what does this mean?

high chances of making a wrong decision. influenced by the design you may even overlook the tech capabilities of the service provider. i personally know of many such errors made by customers.

another, though slightly academic, explanation against this practice is as follows -
as i understand the philosophy of design, ‘form’ intrinsically follows ‘function’. what this essentially means is that the visual design is not merely an exercise of creating the look and feel of a site/application but about evolving an aesthetic perspective which aligns with the ’soul’ of the offering and the sensibilities of the end users. so in order to design, one needs to understand in detail – what are we making, why are we making, for whom are we making, and so on. doesn’t the idea of speculative work contradict this in totality? it also needs a lot of time plus commitment, not just from the designer but also from your team.

don’t we call this a project?

so what is the solution?
work harder!

talk to the design team about their philosophy and solution approach. ask questions. look at their past work, if you are keen to understand the aesthetic sensibility of the design team. but also do remember that the past projects do not merely reflect the aesthetic sensibility of the design team but also of the client for whom those designs were made. haven’t we all heard that a design is only as good as the customer?

what is important is that your designer should understand design, should be able to understand your business, should be able to ideate and should be able to handle tools which would help him/her realize the idea into a working solution. he/she should be your partner who would work with you, to evolve a sensible and working design solution.

let the ‘wow’ belong to the roi!

We’re looking for a Project Manager for working with our development team. Our development process internally is more ‘agile’ – however, an adaptation to our own ways of working. Each Project Manager would typically need to manage 2-4 projects of varying sizes, across 1-2 project teams (4 people per team).

The responsibilities of a (technical) Project Manager at Srijan are detailed out below.

—————–

Business Analysis
Work with the customer in getting requirements in place. Make use of ‘wireframes’ and ‘user stories‘ as an estimation process. Skills in ‘use case diagrams‘ would be a real benefit

Maintain Product Backlog
Work with the customer in creating and maintaining the Product Backlog

Manage Work Plans
Prioritize the Product Backlog, and create ‘work plans‘ or week-long Sprints with the Project Owner (which is essential a Tech Project Leader role at Srijan); Monitor the output, refactor, manage challenges

UI and Functional Testing
Test the deliveries before they are demonstrated (‘victories‘) to ‘internal customers‘ (Account Managers role-playing as the customer); organise demos on our Sandbox/Test environments for end-customers

Sign-off from Customer
Get sign-offs from customers

Scope Management
Manage the original scope estimated for and proposed to customer, vis-a-vis the scope as it changes while the project develops, and the customer becomes more educated in what they desire.

Profitability
Monitor profitability of the project as the project goes along

Manage Contract
Manage contractual requirements throughout the project lifecycle

—————–

Please drop in an email at jobs [at] srijan [dot] in to apply for this opening.

All these years I was wondering, what would be the best practice and configuration to have TYPO3 running huge number records with several connections. I found answers to some of the queries, but still working to get a clean solution. TYPO3 can play havoc if not configured properly. So here are the few tips that should be considered while setting up your server. It is better to get your own dedicated server for a TYPO3 hosting.

Do not use unused extensions

TYPO3 writes information about the every page display. So disable any TYPO3 built-in statistics. The more visitors you get, the slower the site will work if you have statistics enabled.

Remove orphan extensions as some of them provide hooks that executed for every page view. Others do complex actions during TYPO3 initialization phase. Even the simple operation of registering plugins takes time! So get rid of anything you do not use.

do not use indexed_search

Do not use indexed_search for searching web sites until it implements FULLTEXT indexing. FULLTEXT indexing should be faster than current way. But currently indexed_search is not suitable for sites with more than 100 pages. It is terribly slow and puts huge load on the TYPO3 database. Remove it and you will notice how server load drops and performance of web sitre increases.

What to use as replacement? The easiest is to use Google web site search. It can be customized for your web site look and feel. If you do not like to use Google, then I am coming up with an extension which will be integrated with Apache Solr.

use InnoDB for database tables

Yes InnoDB. The main advantage is that it allows simultaneous read and writes. By default MYISAM is used, which locks whole table while anyone is writing to it. So imagine if one user is browsing the page and data is written to the cache table. No other users can browse the site in the same moment.

But do not tend to make all tables use InnoDB. InnoDB is an advantage for tables that are read and written simultaneously. MyISAM is better for tables that are mostly read

optimize MySQL

Every server is different and needs different settings. Even if hardware is the same, sites are still different, data is different and it may need tuning.
But still I will give some tune up configuration which you guys can try out:

query_cache_type=1
query_cache_limit=5M
query_cache_size=100M
key_buffer=200M
join_buffer=4M
sort_buffer=4M
read_buffer=4M
read_rnd_buffer=4M
record_buffer=4M
wait_timeout=15
table_cache=128
thread_cache=32
max_connections=300
thread_concurrency=10
innodb_buffer_pool_size=40M
innodb_additional_mem_pool_size=40M
key_buffer=128M

do not use persistent connections

Firsts, they are not really faster than regular connections. Secondly, resource associated with connection is freed only in two cases: it is freed explicitely in the code or when connection is closed. So if developer forgot to free resource, it will hang alive until connection is alive. This eats memory and eventually may cause swapping on the is level, which always decreases performance. So, go to Install tool and check that you do not use persistent connection. If you are developer, the best you can do is to ensure that you call proper functions to free MySQL resources.

• Try the extension t3p_scalable to enable memcached and database load balancin.
• Use mod_expires, so make sure you enabled mod_expires in apache.
• You can try eAccelerator as a PHP cache. It really boost your performance.

We’re looking for a Projects-coordinator to help us manage timely project communication with customers across all our projects and new business enquiries.

Skills required

• Excellent spoken and written English communication skills
• Good inter-personal skills to be able to coordinate with people across Srijan – approximately 25 people
• Good personal time-management skills
• Training in any software development environment – such as a Diploma from NIIT/similar is required

Roles and Responsibilities

• Track communication (through our project communication tool – called ‘Basecamp’) of all managers and developers with customers, and log Action points that we have committed to with dates
• Track all customer communication to us, where they expect our managers and developers to respond and log Action points expected with dates
• Follow-up with managers and developers to ensure responses go out in time
• Escalate, in case, responses from our end or from customers are getting delayed
• Ability to take care of all Administrator responsibilities and co-ordination would be an added benefit

In case you wish to apply for the opening, please write in at: jobs@srijan.in, with your CV in PDF or .DOC formats.

We’re looking for part-time Project Managers, who can be available onsite for half-day each work day, for managing TYPO3 and Drupal projects.

These managers must have a technology background, although not necessarily in PHP/ TYPO3/ Drupal. They must have excellent skills in

• client communication
• project planning
• task allocation
• release management
• quality assurance
• bug tracking
• tech experience in PHP and Linux sys admin would be helpful

Our development center is in Nehru Place, New Delhi.

Excellent video on the law of attraction of the Universe! Whatever you think of, whatever you are constantly “feeling” in your body – will get attracted to your life. Even Quantum physicists all over the world are saying similar things, apart from Philosophers and Spiritual teachers.

The Universe does not distinguish between good or bad people, good or bad thoughts – it just gives you whatever you are “thinking” and “feeling”.

The “choice” of what you choose – misery or positivity – is yours. If you do not like something about your life – work, home, relationships, money – just choose again! This time stop “resisting” (coz what you resist, persists) and change the “feeling” to the opposite – “having”, “enoughness”, “happiness”, “strength”, “love”.

Getting some of these feelings, when the chips are down, can be difficult. In those times, take some action, which sends a loud and clear opposite message to the Universe.

For instance, when you are having financial troubles, give some money away to the beggars on the road, or in charitable organisations or donate at a temple – not as a manipulative move, but rather with a “feeling of sharing”. If this feeling cannot come by, still go ahead and give. It will come by after a few such actions. Most of you reading this, have enough anyway to share with the bottom most downtrodden people around you – on roads, near temples. Open your heart and share a little bit. See how this uplifts your spirit. Better financial times will flow into your life, with ease.

If a relationship is bad, divert your energy into creating something positive in the world. Give a compliment to someone at work. Divert your energy into some hard-sport. Give. Share a little bit of love with some other people in your life. Volunteer some time in charity.

If work is not going well. Think this is temporary, and will pass. Energise yourself with positive thoughts, and share the negative feelings with colleagues who can help channelise your energies into some positive action-steps, to be collectively taken.

What you constantly “choose”, you attract into your life! The “choice” and “power” to “choose” an alternate, is always ONLY WITH YOU.

Sanitising your code:
Ahm…  Ok adding some techie stuff to your php knowledge. Before accepting any thing from web make sure it is a clean input as you can never trust user input, it may be malicious, So you have to always check your php input. To sanitise this you have to check all global arrays like $_GET, $_POST, $_REQUEST, $_COOKIE, allow only known variables and make sure that they contain the right type of data.

What does this mean ? It means that if you have a $_GET['id'] variable in your script which has to be an integer, always check it and make sure it is an integer.

Also don’t allow other variables in $_GET or other globals, keep only variables that your scripts need. So, if your script only uses only one variable $_GET['id'] then dispose other variables.

Stripping out malicious code
we can scan through the input, searching for anything that shouldn’t be
there, like html code, <script> tags, etc.

———– Code snippet starts ————————-
<?
function cleanInput($input) {

$search = array(
‘@<script[^>]*?>.*?</script>@si’,   // Strip out javascript
‘@<[\/\!]*?[^<>]*?>@si’,            // Strip out HTML tags
‘@<style[^>]*?>.*?</style>@siU’,    // Strip style tags properly
‘@<![\s\S]*?–[ \t\n\r]*>@’         // Strip multi-line comments
);

$output = preg_replace($search, ”, $input);
return $output;
}
?>
———– Code snippet Ends ————————-

’slashing
This part can sometimes get tricky, but not to worry, the code’s not too bad.  Basically we’re adding a backslash before any of the following: ‘(single-quote), “ (double quote), \ (backslash) and NULL characters. Depending on your server configuration, there are a bunch of ways of getting this done.  PHP has something called magic_quotes, which does this automatically.  Note, however, that as of PHP 6 this feature has been deprecated and removed.  Another PHP function, addslashes(), is the manual version of magic_quotes.  addslashes(”Where’s Wally”); will return “Where\’s Wally”.  A better option, if your server supports it, is mysql_real_escape_string().  It performs pretty much the same function, but is apparently better.
———– Code snippet starts ————————-
<?
function sanitize($input) {
if (is_array($input)) {
foreach($input as $var=>$val) {
$output[$var] = sanitize($val);
}
}
else {
if (get_magic_quotes_gpc()) {
$input = stripslashes($input);
}
$input  = cleanInput($input);
$output = mysql_real_escape_string($input);
}
return $output;
}
?>
———– Code snippet Ends ————————-

To use, we simply pass any input to the function. The function works on single strings, as well as deep arrays.

<?
$bad_string = “Hi! <script
src=’http://www.somesite.com/bad_script.js’></script> It’s a good day!”;

$_POST = sanitize($_POST);
$_GET  = sanitize($_GET);
$good_string = sanitize($bad_string);
// $good_string returns “Hi! It\’s a good day!”
?>

Typecasting
Making sure that the data we’re inserting matches the expected type; i.e, someone’s age should be received as an integer value, and not a string.
<?
$age = (int) $_GET['age'];
?>

In addition all these we have “filter_var_array” php function supported for PHP 5 >= 5.2.0, and is available as a pecl package.

One of my colleagues during a general chat said this:

actually for me srijan has become one of those few places where i can get up every morning and really feel like going”
it does not give me a feeling like “i am doing a job, so i have to go to the office no matter what”, that compulsion is not here, thats why i am here at srijan

Now, this speaks a lot for Srijan, and the work culture we have been able to create for ourselves.

I’ll not name the person, and wait till he puts up a blog post about this feeling/experience himself on this blog, which he’s committed to writing. Meanwhile, I wait in anticipation.

Live Mint, the Wall Street Journal and HT venture in India, recently issued a notice to CSE to remove all its articles from its website, and called it “theft” at their blog: http://blogs.livemint.com/blogs/romanticrealist/archive/2008/10/18/whose-ipr-is-it-anyway.aspx.

I am not an expert on IPR issues, but being promoters of open source philosophy for promoting “equity” and “justice” I believe that IPRs can be regressive.

So, a little bit infuriated by the use of strong words such as “theft” and “steal”, I wrote this comment on their blog post.

————

I think the author needs to move away from the use of words like “steal” and “theft”, for the simple reason that CSE has been giving references to all articles it sources from news magazines and journals for its portal, clearly and with links to the sites from where it sourced them. In that sense CSE’s portal is actually “driving traffic” to Mint’s own site, just as it is doing for other sources, and “making them” “earn money”.

Let me also mention that the intended readers of this portal are environmentalists and researchers, who have been extremely under-served in this country, in terms of digitally available research content which mostly gets hidden behind IPR and payment/password protected information. CSE has released its 20 years of “Down To Earth” content, on the portal, with a clear intention of serving this community, and thus enabling the community to help create a paradigm shift in environment research and policy making.

I can say this with some confidence as I have worked very intensely over this year in building the environment portal with Sunita (we’re the development company; and I was leading the project from our end;). People who want to see vested interest in my post here will ofcourse see it, but really I have no business making this post this and defending CSE and Sunita – as we’ve been paid and our ‘job’ on the portal is done. I’ve seen the passion with which she and her team have invested time and energy in minute details of the portal, with the motive of enabling a change in India and this ignored space.

It must have been a big decision for a non-profit which earns nearly all its revenues from its premier magazine (Down to Earth, and related compilations), to release all the content of 20 years on the portal. Ofcourse they want to earn money from the portal (and I suspect they are doing that now, as the author has suggested, as there are no commercial ads running), and should. They must earn well, and carry on their cause of environmental and societal change.

Any such discussion, I believe, if revolves only around rules and policies without considering “motivation” and “intention”, then it ends up becoming inhuman – more like the world we live in today. And isn’t this that we’re all looking to change – all the mess that we’ve created in our world?

I have asked some key people at CSE who’ve been involved in this portal, as to why they are there with CSE, and the answer often has been “this is more meaningful“. Mind you, they are there even while compromising on the higher salaries that they can potentially get at publications like Live Mint. You hear these people’s voice tone change when one talks about Anil Agarwal, the founder of CSE. This motivation and inspiration runs throughout CSE, and is visible in day-to-day interactions with them.

I believe these aspects of an organisation and people, must be taken into account before passing strong judgments on them, specifically with strong words like “theft” and “steal”. I must mention that being a promoter of the “open source” movement in India, we people hate proprietary stuff and promote knowledge and information (and money/wealth for that matter) to be shared among the world’s people for fulfilling a larger purpose of our lives.

Ofcourse, it maybe difficult for people with a pure capitalist (and profiteering only) mindset, to relate this thought process of “sharing” and “equity”, to which most of us in the FOSS community, and according to me, CSE and Sunita, relate quite easily and instinctively.

We realised that we did not have one place where we had pictures of our office. So are some pics from my mobilecam.

The India Environment Portal

URL: www.indiaenvironmentportal.org.in

Platform: Drupal, LAMP, Apache Solr

Key highlights:

• Over 300,000 articles in database
• Migration of data (with massive data transformation and clean-up) from 3 different database sources
• Migration scripts for ongoing data migrations from MS-SQL to Drupal (MySQL)
• Over 8000 tags as part of the Taxonomy
• One of the largest installations of the powerful Apache Solr search for free text searching and tag-based (author and source as well) filtering of articles
• Use of Panels2 with node-queues (almost a mini-CMS implementation) to enable editorial team to create tag-based indepth pages anytime

Background

Centre for Science and Environment (CSE) were inspired to build an environment portal as part of the push for portal development from the National Knowledge Commission, Govt. of India. CSE decided to release 20 years of their research articles spread over 3 proprietary systems, including a Library Management System, a Content Management System written in ASP/MS-SQL for managing the website of their premier Science and Environment magazine – Down to Earth (DTE) – and some Access Databases. The total number of records exceeded 250,000.

CSE also had a very extensive tag vocabulary, or thesaurus, of their library tags, organized as What, Where and Who lists, which represented nature of articles, geographies they represented, and people they were associated with including the authors, respectively. All the articles in its library and DTE database had been classified tagged with these.

Srijan’s initial engagement

1. CSE’s vision of the portal was still evolving, and being a non-profit they had started out with a very limited budget
2. Srijan was engaged with a small Information Architecture exercise for the portal, which included studying of their library management system, the way they tagged articles, and gain an understanding of DTE

Challenges

The DTE content management system had been around for 4-5 years and had grown along the way to meet the changing requirements of the print magazine, without any documentation whatsoever. The database therefore was in a bad shape from a design perspective, with redundant tables and data, adding to the confusion.

Solution

Srijan proposed to break the project into multiple phases, with the first being a pilot for data migration of the DTE database, which formed the bulk of the data, into an open source Content Management System.

Choice of platform

The initial choices were Drupal or TYPO3. We eventually chose Drupal, for two specific reasons namely:

1. Drupal had excellent vocabulary and tag management core modules, which would have to be written in TYPO3
2. TYPO3 had a separate Admin interface which would have proven to be difficult to manage and use for the library team
3. Drupal’s caching mechanism is better than TYPO3; this would be a critical requirement in the high volume traffic that CSE was expecting for the portal
4. A powerful and fast search engine was required for searching through 250,000 articles. Drupal had an Apache Solr search module integrated with it, which was a candidate for implementing this search .

Key challenges during migration

1. Looking at the MS-SQL database we knew that the database architecture was poor with data redundancies.
2. There were some cases where UNIQUE fields were having duplicate values!
3. Database Normalization was missing. Thus the flexibility, data integrity and  efficiency were not so good. Required to safeguard the database from anomalies
4. Since the data was populated through a Library Management tool, the data were having many special characters which were stored in database. During migration of those data to Mysql database, we had to clean up those. It was very much time consuming and repeated work to cleanup the data without the availability of any documentation of the database
5. Designing our database(MySQL) to accommodate the specific concepts of box stories and cover stories that comes with a magazine like website. It required, understanding the relationships of tables and data, the way it was maintained in MS-SQL database. Again, many redundancies/anomalies in the database for these tables.

Finding a powerful and easy way to search

The next phases of the project comprised of setting up Drupal (with a default theme) and build all the required modules to start showcasing the content in the desired manner. A key component of these phases was the setting up of a powerful search.

Apache Solr was researched with, and selected as the choice of the search. It turned out to be an excellent decision, as in retrospect, not even Google Mini (the other search candidate) would have proved to be so beneficial for researchers. In another case study specifically on Apache Solr, we will describe how researchers within CSE are using the same to their absolute delight.

<article in progress; to be completed shortly>

I am evaluating the reliability and response time of different website monitoring services that are freely avialable on the internet.

To begin with I have considered the following website:

• http://www.siteuptime.com/
• http://www.watchmouse.com/
• http://www.pingdom.com

I will soon post the results/analysis of services with the alerts received.

Passion, Work, Focus, Persist, Ideas, Good, Push, Serve. See the presentation to start making sense:

Some words of wisdom from our former President (for a lot of us – especially in the open source world – still our President) – Dr. Abdul Kalam.

(Former President of India APJ Abdul Kalam at Wharton India Economic forum , Philadelphia , March 22 , 2008)

Question: Could you give an example , from your own experience , of how leaders should manage failure?

Kalam: Let me tell you about my experience. In 1973 I became the project director of India ’s satellite launch vehicle program , commonly called the SLV-3. Our goal was to put India ’s ‘Rohini’ satellite into orbit by 1980. I was given funds and human resources — but was told clearly that by 1980 we had to launch the satellite into space. Thousands of people worked together in scientific and technical teams towards that goal.

By 1979 — I think the month was August — we thought we were ready. As the project director , I went to the control center for the launch. At four minutes before the satellite launch , the computer began to go through the checklist of items that needed to be checked. One minute later , the computer program put the launch on hold; the display showed that some control components were not in order. My experts — I had four or five of them with me — told me not to worry; they had done their calculations and there was enough reserve fuel. So I bypassed the computer , switched to manual mode , and launched the rocket. In the first stage , everything worked fine. In the second stage , a problem developed. Instead of the satellite going into orbit , the whole rocket system plunged into the Bay of Bengal . It was a big failure.

That day , the chairman of the Indian Space Research Organization , Prof. Satish D hawan , had called a press conference. The launch was at 7:00 am , and the press conference — where journalists from around the world were present — was at 7:45 am at ISRO’s satellite launch range in Sriharikota [in Andhra Pradesh in southern India ]. Prof. Dhawan , the leader of the organization , conducted the press conference himself. He took responsibility for the failure — he said that the team had worked very hard , but that it needed more technological support. He assured the media that in another year , the team would definitely succeed. Now , I was the project director , and it was my failure , but instead , he took responsibility for the failure as chairman of the organization.

The next year , in July 1980 , we tried again to launch the satellite — and this time we succeeded. The whole nation was jubilant. Again , there was a press conference. Prof. Dhawan called me aside and told me , ‘You conduct the press conference today.’

I learned a very important lesson that day. When failure occurred , the leader of the organization owned that failure. When success came , he gave it to his team. The best management lesson I have learned did not come to me from reading a book; it came from that experience.

Here’s a beautiful presentation on how the power of self-organising groups of people, who are more adept to innovate, will cause a shift in how institutions in the future are going to be structured. Although, it provides an insight on this development, the presentation does not give any answers on what practices institutions should adopt. However, I guess those examples are all over with Ricardo Semler and Semco in Brazil, Scott Bader Commonwealth in UK, Mondragon in Spain, and many others. The key principles, I believe, are (obvious and universal) going to be: democracy at work, love, freedom to express, an environment which allows human beings to express their gifts and talents and equal opportunity for everyone.

The presentation at TED:
http://www.ted.com/index.php/talks/charles_leadbeater_on_innovation.html

We are looking for 1-2 interns with an MBA in IT, who would like to become Project Managers for IT applications in the coming years.

It is important for the candidates to grow into roles defined in this spreadsheet: http://spreadsheets.google.com/pub?key=p7fub_vJsx_uk7OIVuIEnew. We’re also looking at interns from a long-term perspective. So, apply only if you feel you can have a longevity of 2 years, including your training period.

If this is interesting to you, please send us your CVs at jobs@srijan.in, with the subject marked “Project Management intern”.

We’re still looking for a Team Leader / PHP Tech Architect / PHP Project leader role, with about 2-4 years experience. We must have interviewed atleast 25-30 odd people from small to large sized ocmpanies over the last 2 months, and not selected even one, so far.

We’ve rejected people with over 3 – 4 years experience, as their basics have not been clear, and yet they claim to (and truly are) building big live projects. I have no clue how these companies maintain quality, or deliver projects that perform well.

There seems to to be very little desire in people to excel these days. It is so difficult to find good people, with good PHP skill, and with a lot of enthusiasm to learn, and sometimes put learning over money. There are some good people, I must say in organisations similar to Srijan (or sometime larger) but they are not looking for a switch.

There is a job post we made on Monster India : http://jobs.monsterindia.com/details/6134286.html?sig=js-1-d8e2a0f1c7a23626210cc6677936c9f2-1. Hope this helps.

Enough ranting for today…

A rejoinder to my earlier post. Here’s a video of the CEO of WorldBlu, with a beautiful introduction or organisational democracy.

Here’s the The WorldBlu List of Most Democratic Workplaces™ 2008. WordlBlu’s mission is to have 20,000 democratic organisations globally by 2020. Wonderful idea! However, I think their goals are underplayed a bit. In 2009, we hope to have Srijan find it’s way there in this list.

The rise of collaborative networks

At the rate collaborative networks are being formed, I feel this goal of 20,000 organisations is a relatively low target. Besides 2020 is way too off. If we truly desire to create a just and more equitable world, and avert this serious climate change crisis that looms large on our heads, the way corporations function would need to change much sooner and in much larger numbers.

New models of enterprises are emerging, inspired by successful examples of human networks such as Wikipedia, and a whole range of organisations and organisational democracy evangelists and advocates speaking out more and more, and reaching a larger audience. Some such individuals and organistions that I know of are:

• http://www.ted.com/index.php/themes/the_rise_of_collaboration.html
• http://www.eei-consultants.com/

Just changed the theme from a dull boring corporate Blue, to this new colourful one (it’s a little bit of an overdose of colour, I must accept, yet a very good one). Now that is well-suited to represent the “Spirit of Srijan“.  Also, this one is much better organised and visually tight.

I cannot seem to understand the logic of these “Secret Questions”. I have forgotten my password, and they expect me to remember some silly secret question like “Who was your childhood hero”.

Isn’t this a serious usability issue?

What a mess!

We’re intending to start a service to manage content – text and images – for businesses running TYPO3 websites.

Such services are difficult to find in Europe and USA (and other western countries), as it costs hiring skilled TYPO3 operators maybe quite high. These services are required infrequently by companies, and often they do not want to spend their manager-time in doing these regular updates, specifically, when they are related to managing graphics as well.

The website promoting this service will be www.typo3support.in.

Currently, we’re intending to learn a bit about our potential customers, and therefore have put up a survey form, as advised by Internet Marketing specialists – Andrew and Daryll Grant.

As per client project requirement, I had to do some string matching …… and behold, Ahm… that too keeping the pattern intact. So googled through the few pages, then suddenly came with an idea to give a try to regular expression [regexp].

Now who is going to help me out. Bingo!! take help from PJ [Our Sys Admin]. Hey PJ can you help me out with the problem. Sure , he forwarded with a link and page 192 of the OReilly book for perl. Hmmm… now i got some idea. Tried to combined both googled page and reference from the book and that resolved my worries.

Challange:

<ab-cd>Some text</ab-cd> <de-fg> Some text</de-fg> …..

Take out all the tags and keep the text between the tags in order.

Solution:

[php] preg_match_all

$pattern = “/(<([\w\-]+)[^>]*>)(.*)(<\/\\2>)/”;

preg_match_all($pattern,$content,$out, PREG_PATTERN_ORDER);

$out[2] will display an array of matched tags

$out[3] will display an array of text with the matched tags.

Writing this very inspiring post as a lot of us sit at Centre for Science and Environment, working on our overnight mammoth effort to get the India Environment Portal ready for launch this afternoon by Mr. Sam Pitroda, the head of the National Knowledge Comission (we heard that our Finance Minister – Chidambram – is also going to be present for the event). I am simply here to be a “moral support” for Ipsita, Syed and Shashank, as they plough their way in getting the front-end visual look organised, and after long sessions of networking on LinkedIn, I am now writing this blog post. Here goes…

My calling

This afternoon I was with a friend, an IIM-B grad, to get advise on a new initiative I am looking to get moving on at Srijan. Having engaged him in my passionate talk about what I would like to Srijan to become, I shared that I would like to at the nearest opportunity like to move out of heading technology services for Srijan, and help diversify this company into areas that are “my calling” – renewable energy, rural tourism (and other businesses which would transform the social and economic fabric of rural life in India), and pursuing purely social goals of being able to strengthen the communal, social, ecological, and political setup of this country.

I love what I do at Srijan. I love consulting on projects like the India Environment Portal, where we have helped drop three proprietary systems and replaced them with a single open source system based on Drupal (more on this post-launch as a detailed Case Study). I love the idea of creating a company which has a great co-operative way of functioning. Today evening, I am to start discussions with an Investment Banking company to start Srijan’s journey towards creating an an ESOP model as a way of institutionalising our profit-sharing mechanism – a step towards a more democratic and equitable model of running companies with serious people participation and ownership. Having done this, I would like to move out of Srijan’s technology operations to pursue other passions and my other callings.

Towards the end of the conversation, a review of a book that I had read a couple of weeks back, called “A Year to Live“, popped-up in my head. I had actually started yesterday morning with this thought and walked in our park trying to internalise and firm up my resolution to live this one year, as it were my last – and in the process rejuvenate my relationships at home (which I have sacrificed and neglected a lot for Srijan), to push forward all the things I would love to see at Srijan, to “leap” forward in pursuing my heart’s inspirations. I shared with my friend, how powerful it would be to live from a place where one had just one year to live. He mentioned that Steve Jobs had lectured at Stanford on a similar thought.

Steve Jobs’ lecture at Stanford : Trust in yourself; Love Live as if it were the last day of your life

Googling for Steve Jobs’ lectures, I finally found this one. It was so inspiring that I could not hold myself back from writing this blog post at 4:00am (while my colleagues are busy punching the keys on their laptops, with me giving them “moral support”; and one of our client’s managers snoring behind us ). Here’s what he said – some excerpts.

Do what you love

Your work is going to fill a large part of your life, and the only way to be truly satisfied is to do what you believe is great work, and the only way to do great work is to love what you do. If you haven’t found it yet, keep looking, and don’t settle. As with all matters of the heart, you’ll know when you find it, and like any great relationship it just gets better and better as the years roll on. So keep looking. Don’t settle.

Notice that he says “love what you do”, and not “do what you love”. But to be honest, having read the context in which he says this, I think he really meant “do what you love.”

Trust in your gut, destiny, life, karma

You have to trust in something–your gut, destiny, life, karma, whatever–because believing that the dots will connect down the road will give you the confidence to follow your heart, even when it leads you off the well- worn path, and that will make all the difference.

Live as if this were your last day

When I was 17 I read a quote that went something like “If you live each day as if it was your last, someday you’ll most certainly be right.” It made an impression on me, and since then, for the past 33 years, I have looked in the mirror every morning and asked myself, “If today were the last day of my life, would I want to do what I am about to do today?” And whenever the answer has been “no” for too many days in a row, I know I need to change something. Remembering that I’ll be dead soon is the most important thing I’ve ever encountered to help me make the big choices in life, because almost everything–all external expectations, all pride, all fear of embarrassment or failure–these things just fall away in the face of death, leaving only what is truly important. Remembering that you are going to die is the best way I know to avoid the trap of thinking you have something to lose. You are already naked. There is no reason not to follow your heart.

…Your time is limited, so don’t waste it living someone else’s life. Don’t be trapped by dogma, which is living with the results of other people’s thinking. Don’t let the noise of others’ opinions drown out your own inner voice, heart and intuition. They somehow already know what you truly want to become. Everything else is secondary.

For me personally, a year to live is more “realistic”, than living as if it were the last day of your life. Either way, the purpose of this message remains the same.

I cannot but help reiterate the key messages once more:

Remembering that I’ll be dead soon is the most important thing I’ve ever encountered to help me make the big choices in life, because almost everything–all external expectations, all pride, all fear of embarrassment or failure–these things just fall away in the face of death, leaving only what is truly important. Remembering that you are going to die is the best way I know to avoid the trap of thinking you have something to lose. You are already naked.

There is no reason not to follow your heart.

The full lecture, which I highly recommend everyone to read, is available at: http://www.interesting-people.org/archives/interesting-people/200506/msg00229.html.

Let me know if you found his talk inspiring as well.

A few days back we got in touch with a customer who had stopped working with us about 3 years back. After the initial pleasantries, he wrote back asking me the following:

What would you say has improved in the past couple of years that’s
enough reasons for our company to choose Srijan as its outsourcing
partner?

Can you send me the latest processes and rates?

What all went wrong? What all was broken at our workplace?

The customer had stopped working with us (this was about 2-3 years back in 2005-2006) for several reasons including delays in delivery from our side, lack of overlap hours for him to manage projects effectively (he’s a very smart application architect, programmer, sales person, and manager – all rolled into one – and since he had to work with our team closely in odd overlap timings he was certainly getting stressed), our Project and Resource planning problems (lack of skills and tools), and also too many overlap projects which led to shuffling across projects.

We were just about get strong on our feet with TYPO3, and all of a sudden we had these BIG projects coming our way, which we ofcourse took on, miscalculated the effort estimate, and then there we were calling on our best developers to develop across both projects shuffling between the two and somehow hoping without plan to be able to deliver them both.

We had delivered several TYPO3 projects till then and quite successfully as well, but primarily they had been all small ones.

Anyhow, we delivered the projects in good shape and they are live, but the customer had moved to explore another outsourcing setup in East Europe – partly also to have better overlaps with US timings, apart from a dedicated team.

Life’s about “becoming better”!

Here’s what I wrote to our customer:

Improvement and “getting better” is a continuous goal for us, as should be for any individual or company. We’ve constantly been striving to improve all these years – technically, in terms of methodology and processes, and with our people values and practices.

False propaganda by companies

Truly, if there is one thing we can say we’re striving for in our lives – it is to become “better human beings“.

Why should this be any different for companies? Why must companies find it difficult to acknowledge their weaknesses and continue to write about themselves in fantastic flowery language, about how good they are and how great there processes are. Why must customers (and prospecting employees) expect companies to share such false propaganda on their websites, and not accept their “human-ness”, and that they do make mistakes, but are willing to work on them, together with their help.

Responsiveness and “continuous learning” is most important

I was speaking to my coach recently, and mentioned that with God’s grace, we now have a team, which gels very well together, and that we have excellent chances of doing very well, as we are willing to work together with each other and towards common shared goals. He successful team must also be willing to “work together” and “learn together“. How true! Is it not?

Learning is about continuously improving ourselves, learning from the mistakes and challenges that we face. It is about “responding” to problems, effectively and in time (immediately, or, as soon as possible).

Srijan, I must say, is responsive to it’s customers and associates. We may falter, but as a practice, we’re learning to become better day-by-day – and together!

What is right about Srijan?

Let me explain what all has changed at Srijan.

• We’ve gained much deeper experience in developing applications on TYPO3 and on other open source frameworks. We do heavy duty projects and products on Drupal, Symfony, Flex, Ruby on Rails and Django/Python.
• This has been possible with “people” carrying on with Srijan over the years and passing on the knowledge, owing to us being able to create win-win situations with their career, monetary growth, and treating all our people with love and respect.
• We’re constantly evolving our structure, and methodologies to make timely project deliveries and of better quality. For instance, we started using SVN quite some time back, and are currently working out a way to have a DEV-QA-PRODUCTION setup so as to ensure that developed code passes through a formal QA process before going into Production environment for client demos. This is a milestone based release, so customers keep checking application development as the work is progressing. The unique thing is that such an environment which is typically forced (and thus taken for granted) in Rails applications, is being implemented for TYPO3 as well – and we do not know of many companies following such a process – even in Europe.
• Our number of Project Co-ordinators/Managers has increased; thus developers and Tech managers do not have to manage requirements, quality checking, etc. and can focus on development and mentoring their co-developers
• We’re about 20 developers+managers team now, with about 14-15 actively in PHP (TYPO3, Drupal, Symfony) related projects
• We’ve invited a Drupal guru, as a consultant, to mentor our team on Drupal on a weekly basis, thus helping get past the learning curve, instead of struggling over the same ourselves leaving a trail of failed projects behind. Thanks to Jacob Singh – http://www.civicactions.com/team/jacob_singh – is an American living in New Delhi, and one of the leading contributors to Drupal, and a fantastic “trainer”, with over 9 years programming in PHP, Python on a host of applications; having managed $500,000 projects with small teams. He’s expensive for us in terms of money now. But for the success of Srijan – he’s god sent.
• The key element here is our real big push for internal trainings. We have a sort of an excellence group, which is training all others (and each other) on – software development best practices, TYPO3, Drupal, CSS, Ruby on Rails and Adobe Flex
• We’ve picked up competence and skills in Adobe Flex, and are writing our first BIG project. This happened after a LOT of time investment in our very loved (and rather, spoilt!!) Interface Architect, who is passionate about anything on front-end. What we now produce is not just ordinary stuff, but some top-notch work on Flex
• We’ve got a partnership going with a German company – www.crocsystems.de – who is now a major customer for us, and a very valued partner. We’ve made visits to each other in the last 6 months, and are constantly working on getting this partnership to work for both of us. This is a leap forward for us, as we’re now known to European customers when we build projects – and a climb up the value chain for us
• We’ve developed competence in Ruby on Rails and in Django/Python (although the latter remains restricted with only a few people, and very little focus from us)
• Also, we’ve diversified into writing some products, including some excellent work using GPS devices; please see http://blogs.srijan.in/author/buckycat/
• We’re working on institutionalising our profit-sharing methodology, and making our Project Reviews a simpler and more regular – this allows us to continue with our stated goal of equitable and participative growth for all of us at Srijan
• We’ve made a group of 5 innovators, who will have both our working Saturdays each month, off for them, so that they can together spend time in researching or building interesting product ideas – our own customised version of Google’s research culture
• And here’s a very BIG one for us. All projects are being planned in Srijan now using a very nice multi-project planning tool called ConceptDraw Project. It’s been a boon for us, in terms of providing realistic timelines to customers on starting and completing projects. This has been a big pain area for us earlier.
• We’ve also increased our price of working to EUR 25 to EUR 30 an hour (for Europe; we maintain variable pricing depending upon economic situations of the regions) thus allowing us to avoid overloading people in our requirement to keep our cash flows going. Therefore, we now actively refuse projects, even while maintaining healthy profitability.

Ah, so we did away with all our problems?

All this does not mean that we do not have problems.

We still have problems of Resource planning and allocation, lack of proper planning and incorrect estimations, sudden spurts in projects leading to people getting overbooked, people falling ill (etc) throwing tight project schedules off-track, developers having to switch between projects, etc. – BUT we have LESS OF all these problems; AND we are CONSTANTLY working on IMPROVING practices and policies to make them REDUCE FURTHER.

What does all this mean to a customer?

We’re responsive to circumstances and situations; and ever more responsible to our customers and our associates!

We have a Project Manager opening at Srijan, for managing a team of TYPO3 and Drupal developers.

Now, in order to define the role and skills, as is traditionally done in a job description format, I decided to write down User Stories – what innovative adoption of agile methodologies to look for for a Project Manager . No kidding – this was way easier than writing long sentences.

Here’s the Google spreadsheet defining the opening: http://spreadsheets.google.com/pub?key=p7fub_vJsx_uk7OIVuIEnew.

If you’re interested and feel you fit the role, please write to us at jobs@srijan.in, with the heading “Project Manager : TYPO3 and Drupal”.

Here’s a preview on how our Drupal training is proceeding with Jacob Singh. After a day long session hacking around on Drupal, we spent some time understanding best practices to follow in case of websites with large amounts of data.

slick professionalism

One of our clients wanted a shutdown scheduled one night at 10pm. This was a suitable job for the “at” command, and I set it up.

Then I noticed the time on the system was out by several minutes. (This was nothing to do with what I’d just set up with the “at” command – this was simply due to hardware clock drift. This system had been running for over a year non-stop – a record we were proud of. But, yes, I reckoned I should fix it instead of having the client wait a few minutes extra past 10pm, twiddling his thumbs. I’m a professional, don’tcha know?)

There are tools and things to handle time drifts (adjtime, ntpdate, rdate, hwclock) but since I did not want to change anything on the system – this being a production server – I decided to be cautious and use the closer-to-the-bare-metal “date” command. Slick professionalism from me, indeed.

slick stupidity

So, I typed in the time according to what I thought was the correct syntax using date. But the moment I pressed the enter key, the system suddenly thought it was about 5 years into the future. No problem, I thought, I’ll just read the man page for date and fix it.
But, a few seconds later, the system (having gone through the “at” spool in its usual once-a-minute run) decided the “at” shutdown job I had scheduled for later that night was about 5 years overdue. And to my horror sent me a shutdown message.

All my smug professionalism circled the drain a few times along with the debris of messages from services that neatly shut themselves down. The system sank in front of my eyes as my fingers reflexively and futilely played a tune of despair in the key of ctrl-c. I don’t remember much of the tune, but the lyrics went something like:

Arghhhh. Arggggh. Arggghhhh.

You may sniff at my melodrama, and I would normally agree that a simple reboot would have sorted things out, and so it would not have been a big deal, but

1. the machine was a remote machine,

2. belonging to an important client and

3, worst of all, I’d just configured the machine to boot up next time in an unreachable state (for a new network).

Talk about cooking your own goose.

Calling up the client to explain what had happened and how to fix it was kind of sucky.

Post mortem::the tech error

I went through the man pages of date, to see where the technical part of my error was. It turns out that the date setting string for BSD/unix and linux differ.

For reference, this example date:

Sat Jul  5 12:42:00 IST 2008

is set by:
date 0705124208
in linux.

and
date 0807051242
in UNIX/BSD

Who’d have thunk it?

(Tip: The linux version can also takes this format if you want to synchronize the seconds field:
date 070512422008.58
Sat Jul  5 12:42:58 IST 2008 )

And the moral of that is: Don’t let the date command spank you on the ass on the way out.

Post mortem::the processes error
Still, this aspect of the error is just the technical part, and is just one of those things a sysadmin can run into.

A more important aspect is the process part of the error. What was the process that allowed such a technical error to get through?

When the Nobel Laureate physicist Feynman was called in to help troubleshoot why the Challenger space shuttle blew up, he pretty quickly cut through the technobabble and showed very convincingly with a tabletop experiment how an o-ring had failed due to the cold. He explained that, yes, the o-ring was the direct cause of the disaster.

But more generally, he emphasised that there was a management culture failure that allowed the technical failure to happen. He spent most of the report explaining that this was the real failure.

While not exactly analogous, where I went wrong was running the date command on a production system when I should have tested it on a local system first. Definitely a process issue gone wrong. Everyone is tempted to take shortcuts to save time, and most of the time we get away with it. But on a production system you need to make sure everything you do is correct.

So the moral of that is: test even the most mundane things thoroughly before running things on a remote production system. Yes, it can take 10 times as long as a result of that, and it may seem hard to justify it. But this way you save yourself getting your ass spanked by the door on your way out and toppling headfirst into that inviting pile of manure as you head off on to the road to salvation.

Pardon my mixed metaphors.

PJ

Just finished watching the movie “aninconvenienttruth” on Global Warming. Almost feel like jumping outright into taking some hard steps from Srijan, about pitching in to solve this crisis. This is not the first time I’ve learnt about the environmental crisis, but it has never come out so beautifully, so clearly, and so very starkly.

The concern of the next project, and the next deliverable for some reason still hold precedence. It’s almost like the quote shared in the movie “It is difficult to get a man to understand something when his SALARY depends upon NOT UNDERSTANDING IT”. While this statement is made in the context of scientific reports being twisted to achieve profiteering objectives of a few powerful lobbyists, I guess, it holds somewhat true for people in business such as Srijan.

If the whole civilisation is threatened, it must be crazy to think of the next project and the next deliverable. There will be no one left to do business with, no one to sell to, if we as a civilisation, continue like this; concepts of, and importance of money would be altered forever; lifestyles would be changed; survival may become the most important priority.

Yet that remains the reality within which we must continue to work, to run our homes and our families – NOW. So what could I do, once again, in the NOW? What can Srijan and Srijan Foundation do? These are questions which are running wild in my head at the moment.

I guess the answer lies in taking some key initiatives with a little more sense of urgency, which I’ve been planning but procrastinating. Most of these would revolve around “increasing awareness” and thus helping to bring about “consciousness change”.

After several months of conceptualising, discussing, wireframing, and delaying, we finally started development of a pet product for the Tourism industry. Today, early (Sunday) morning, I create a product Blog and completed five posts on the same. It’s currently Private, and we will make it Public once we have the product in some decent shape.

Meanwhile, I wanted to write this blog about how we are using SCRUM to final move our asses on this product.

With Jacob using the SCRUM approach to build this product, we agreed to write a some user stories defining  what the different roles – “Site Owner” (the company who buys this product from Srijan), “End-Customer” (end-user of the product), and “Srijan’s Admin” – would be able to perform with the website.

Having done that, the team discussed the goals of the project, and in this process the stories were breaking down further. This has been a very stimulating exercise for me, as I was being asked to set “specific” “measurable” functional priorities for deliveries with Sprints of one week each.  The idea is that we would take only very few set of things to achieve till the next Sprint - which was to be the next week.

Now, I had shared the PPT made by our Information Architect – Navin Pangti – with Jacob, but I myself was unsure if that is what we wanted from the product, as we had changed the concepts a lot during earlier discussions, and prepared a Tag based approach, and those changed concepts were applied in the travel portal we are building for a client on TYPO3.

However, the ideas we had mulled over and over, during the last months, had a few things which could be termed as “core of the application“. Now, for the first time, I was prioritising the core functionalities of the application, to as much detail as I could visualise – and to be honest, I am pretty bad at being able to visualise.

After the first Sprint, during the session in the second week, the great part of this exercise was that I, as a customer (Project Stakeholder) I could now “see” (and use) how the “Site Owner” would be able to create the key desired functionality from the product.

Even after having done an extensive HTML wireframing exercise earlier, the functionality required and the priorities of development, were not as clear to me as it was now. Setting priorities for the Second Sprint, was an exciting feeling. The first time this product, in almost an year, is becoming a reality.

Jacob actually suggested that we should now soon have some Visual Design in place to help the developers define the template layouts. My argument was that we have done an extensive exercise in Visual Designs and HTML wireframes, and still never had real clarity on how this product would be built and how it would bring the value, we so want it to deliver to travel companies – and that this approach without too many wireframes and designs, and simply working with out-of-the-box Drupal modules (as much as possible) was a better exercise.

Ofcourse, he had a point – that developers need to visualise where and how the desired content needs to be displayed. To be honest, I was being able to define all these priorities and core funtionalities, because, we had done a LOT of exercise in defining the core concepts on the White-board, in designs, and finally in well-designed HTML wireframes.

So, I guess, a dual approach of HTML Wireframes, and rapid prototyping using a CMS like Drupal seems to be a great way to build products – products that the promoters/sponsors also do not know enough about during the the early stages of development – when they are in a place toying with a very interesting idea, but have far too many gaps and no specific way the product could shape up.

SCRUM, and the idea of writing and prioritising user stories, for each Sprint, week after week, and staying completely involved in the product with the development team, is seeming to be a great way to build products. For sure, however, it needs patient sponsors, who have complete faith in the skills and ethical business practices of the development team.

Introduction

For files a few MB in size, users can do backups to their google mail using their local cron.

“Few MB”? What does “few” mean?

Well, we have a back-end that uses Google mail, and google has 20MB as their message size limit. So we’re talking around 20MB, less the encoding overhead. MIME-encoding, the encoding standard for this sort of thing expands a file. So in practice, the file size limit is probably about 17MB.

Why have a limit anyway?

This is because mail is a terrible way to do file transfer (google for more on this). So having some limit is a good thing to run into. Remember, every time you do a file transfer with email, a techie cries.

Fixing postfix to handle larger attachments

OK, so google allows a message size of 20MB. What about at the other end, where the mail is sent to google?

On our development box, we use postfix, which has a default of 10MB. To crank it up to 20MB, add:

message_size_limit = 20480000

in main.cf. (I placed it just after the mailbox_size_limit line).

After a postfix reload, postfix on our box, too, can send a 20MB-sized message.

Now the remaining part becomes an exercise in userspace scripting for cron:

Sending attachments from the command line to your mail

Here’s the script for doing a backup of the current datadump

#!/bin/sh
#backup to a file, unimaginatively named backup.tgz
tar czf /home/pj/backup.tgz /home/pj/datadumps

#Mailing a tgz file to google won’t work because google doesn’t like those kind of attachments

#(http://mail.google.com/support/bin/answer.py?answer=6590)
#solution: encrypt the file to be mailed

#(with the kind of password an idiot would put on his luggage is good enough)

gpg -c –passphrase 12345 /home/pj/backup.tgz -o /home/pj/backup.tgz.gpg

#Let’s time stamp it for convenience:
TSTAMP=`date | tr ” ” “_” | tr “:” “_”`
mv /home/pj/backup.tgz.gpg /home/pj/backup.tgz.gpg.$TSTAMP

#Sending it as an attachment: Use this construction:

mutt -s “today’s backup” -a /home/pj/backup.tgz.gpg.$TSTAMP pj.spambait@srijan.in < /home/pj/some_bodytextfile

#some_bodytextfile is just a file with some text that will go in the mail body.

#pj.spambait@srijan.in is your destination sender address

Put the above in a daily cron and you have a daily backup.

Caveat: add $EMAIL to the user and cron environments if mutt still won’t work for you:

export EMAIL=pj.spambait@srijan.in

That’s it. Offsite back up with minimal effort.

PJ

How would life be, if we made “better choices” everytime.

In today’s session with my coach, this point came out very powerfully for me. We have a choice in whatever we ‘do’, ’say’, or “be” – in a conversation, in a relationship (with others, and ourselves), in our work, in our general attitude.

What if we constantly asked ourselves “is there a better choice I can make in relation to _________” (fill the blank with any current situation, circumstance, etc.)

Just wanted to share this, to me, a very powerful way of living.

We came across this brilliant presentation as a case for Drupal, specifically to be used in institutions and in libraries: http://www.vimeo.com/991273. This has a comparison between the proprietary CMS Fatwire and Drupal.

There is a brief intro to Apache Solr working with Drupal including some sites where it is in production. We’re looking to use the “Apache Solr” module with Drupal for the “India Environment Portal“, coming up for the Centre for Science and Environment.

Buoyed with the growing work pressure and peer support, Ritesh had planned to create his clone….however since he couldn’t spare the required time to create a proper clone, he ended up with creating a monster…

Now it has become a community project to send this clone back to the depths of hell…else he may turn out to be Frankenstein monster… Well we seriously miss “Hellboy” atleast we would have had someone to fight this monster…

Ritesh don’t worry we all are with you….we have the doctors and ambulances ready for all emergencies…

God Bless You!!!

The pictures of postures uploaded below have been done by experts or under the guidance of experts. These must not be imitated or tried without proper supervision or guidance. But definitely not at your offices.

Customer Support – Srijan style

Intercom – Srijan style

Balancing the long legs

More to follow… keep watching the space…

“LY” These are just two letters that give an absolutely different meaning about the concerned person. Following are the pictures which were taken during a marathon working hours. The pictures were taken after working on a project for almost 14-18hrs a day continuously for a week(even on Saturdays and Sundays).

Here’s congratulating the bravehearts…

More snaps would be following….

The Dark lord.

Intro

When we monitor a site we usually think of looking through logs or a gui front end like munin to look at how the server is doing. But if the site itself is down, how do you access this data?

Well, you could figure out some way to do this (a common method is to pipe the logs to another location so you have a chance to diagnose things when they go wrong). But most of the time you just want to be alerted and run basic remote checks on failure to narrow down the possibilities.

The most common things to check remotely should be, in my experience:

1. Is the webserver responding?
2. Is the response ok or garbage?
3. Is the connection to the internet broken somewhere along the path?
4. Is the DNS system broken?

You can add more, but this seems to be the most useful stuff you want to have a look through remotely.

So, how do we implement it ?

To do the job, you can roll out a heavyweight nagios or whatever. Or you can go the lightweight route and script something customized up – that’s easy too, but also invariably ugly. I like lightweight stuff, but getting something pretty out is better for future maintenance for when I leave srijan.

Fortunately we are spared such ugliness because there is a free web service (free-as-a-service) with a nice GUI interface to do it all for you. So monitoring becomes SEP (with the only catch being that they gather data from the server diagnostics over time).

Basicstate

The service is called basicstate and is pretty slick. Plus they had a responsive and clear customer support for some questions I had.

There are two problems with this service though.
1. it doesn’t (yet) offer SMS alerts on the server being down for Indian users (the SMS gateways they provide only seem to work sporadically, and if they do work, the delivery delay is often 24 hours.
2. if we did send an SMS alert in showing a problem, by the time the alert is actually noticed the server may be back up/reachable again. So effectively we have a false positive – though I should point out it is not really a false positive – just a result of excess sensitivity for our purpose. Using the basicstate web interface, we can check if the fault persists for 15 mins or more, which will filter out most of the false positives. But we can’t check if the fault persists for a time of less than 15 minutes (using the basicstate interface). It is not such a big deal, because most people don’t need that short a persistence-timecheck, but I decided it would be a useful feature to have for diagnostic reasons, especially for our more critical sites.

We can solve both these problems (no sms service and too long a gap between detection and follow up diagnostics) with procmail and an SMS delivery specialist.

Webtext

Webtext is an SMS delivery specialist, and like basicstate they have a slick interface and a responsive staff. They aren’t free though. But they are pretty low-priced for low volume SMS service (I’m thinking of 100 SMSs in a year).

(Aside: On choosing outfits that are small, like basicsense and webtext: My experience has given me a lot of faith in the classic Avis principle of smaller outfits trying harder, and therefore often having better products).

Where were we? Oh, yeah, I was covering how I was fixing some problems with basicstate, namely by adding tools from webtext (the SMS service) and procmail.

Procmail
Procmail is a old mail filter tool whose capabilities I’ve grown to respect. I can’t say I like it much – its recipe language is a bit strange, and you have to do some head scratching about odd quirks sometimes to use it (De Morgan’s theorem to implement or-like behaviour with ands and negatives is a particular favourite of mine that I wheel out of the recycle bin of my memory most of the times I use it). But yeah, it gets my Respect (note the capitalization) because I’ve seen it handle crazy loads and remain standing when all other contenders get bogged down and croak and flap their arms around uselessly.

Fortunately there is a lot of excellent material on the net that covers how to do procmail recipes (including the de Morgan trick and its ilk). Nancy McGough’s procmail quick start is a great start (http://www.ii.com/internet/robots/procmail/qs/)).

Joining the pieces

So, how do you combine all these beasts?
1. get your mail from basicstate handled by procmail
2. in procmail, during the filtering, run a diagnostic immediately on the site that is having a problem (webserver, DNS, traceroute)
3. still in procmail (if the above diagnostic showed a problem) wait a little, say 3 minutes or whatever you decide, then run another diagnositic.
(i) If there is *still* a problem, then send an SMS to the sysadmin’s cellphone via webtext, and also a copy of the diagnostic results to the sysadmin email account.
(ii) If there is no longer a problem, forget it.

Conclusion

There you are – a lightweight remote monitoring system with SMS alerts and fine tuning to the drop level for false positives, and diagnostics implemented the way you like. How splendid. It means you don’t end up red-eyed and bleary and sharpening your axe so you can slaughter your colleagues the morning after.
It’s working pretty well. Most failures are network connectivity failures, one of them seems to be a webserver overload failure (“seems” because I have yet to examine the issue in detail, since the junior sysadmin should have got off his pimply ass sometime ago to have a look at it, but hasn’t). Anyway, this simple monitoring system has probably saved me a lot of eyeballing of logs and graphs and lets me be pretty much the first to know when something is wrong – which is as it should be.

PJ

Hi All,

Our colleague, Chandraji(fondly called Prabhuji) has co-authored a spiritual magazine by the name “Anand Dhara”. At the moment its in Hindi version only. Translations to english and other languages is in process and once this is done will be updated, its a good read.

The most important aspect of this magazine is that, its been co-authored, edited and printed by Chandraji, he himself being a hardcore techie and had actually taken time out for such a not-for-profit cause.

Please click here to view the e-version of the magazine.

On behalf of Chandraji,

The Dark Lord.