Sering sekali saya membaca postingan lowongan ttg evaluasi program. Beberapa organisasi ada yang menuliskan metode-nya secara detil, sebagian lagi secara samar-samar, dengan harapan bahwa si konsultan sendirilah yang akan membangun metode tersebut. Hal ini sah-sah saja mengingat konsultan terkadang telah memiliki alat tersendiri dalam mengumpulkan dan mengolah informasi. Berangkat dari pengalaman saya sebagai konsultan evaluasi, baik sebagai konsultan independen maupun bekerja dalam tim, berikut ini ada beberapa tips yang bisa bermanfaat besar untuk mendapat hasil yang lebih baik:
1. ToR
Lembaga yang bersangkutan biasanya memuat ToR baik yg umum maupun spesifik, ada baiknya selain ToR, tersedia link mengenai program proposal, laporan program, teori of change yang digunakan/ logical framework approach, garis besar training materials bagi penerima manfaat serta fasilitatornya. Hal ini akan mempersingkat waktu konsultan dalam mengembangkan alat yang dipakai untuk melakukan evaluasi. Laporan program sangat bermanfaat terutama memberi gambaran awal akan data-data yang ada serta berdasarkan hal itu bisa menggali lebih dalam akan keberhasilan/ kegagalan program.
2. OEDC kriteria/ ALNAP website
Saya pernah mendapat tugas mengevaluasi program dimana konsultan tidak diberikan output yang spesifik, dan konsultan lainnya yang bekerja untuk proyek yang sama tidak memiliki framework yang jelas sehingga malah membingungkan klien akan hasil evaluasinya. Jika program manager kesulitan membuat output maka dapat menggunakan OEDC criteria dan mengunjungi laman ALNAP untuk membuat panduan mengenai evaluasi tersebut. ALNAP menyediakan platform yang memudahkan semua pihak (program manager hingga konsultan) menggunakan framework M&E
3. Evaluation Framework
Pastikan NGO anda mengembangkan evaluation framework bagi setiap konsultansi atau minta konsultan untuk mengirimkannya (subject to approval dari klien) sebelum terjun kelapangan. Jika tidak bersiap-siap dengan kejutan yang akan dijumpai di laporan nantinya.
4. Be Flexible
Terkadang tidak semua hal dalam LFA harus di evaluasi, terutama jika dana dan waktu terbatas. Negosiasikan dengan konsultan mana hal-hal yang paling penting bagi organisasi anda untuk di evaluasi. Dan bagi hal-hal yang tidak tercakup, usahakan datanya anda dapatkan dari monitoring reguler atau laporan berkala.
5. Budget
Pastikan dalam setiap lowongan konsultansi, organisasi ybs menyebutkan angka nominal pekerjaan tersebut (budget), sehingga konsultan bisa menimbang apakah akan mengajukan bidding atau tidak, hal ini akan menghemat waktu anda juga dalam menyeleksi kandidat sesuai budget yang anda miliki.
6. Respect
Organisasi yang baik tentunya akan menghargai siapapun. Perlu diketahui bahwa untuk membuat proposal terkadang dibutuhkan waktu minimal tiga hari bagi konsultan untuk membuat proposal. Sering, proposal dikirim tetapi tidak ada kabar dari organisasi penerima. Tunjukkan bahwa organisasi anda menghargai usaha mereka dengan mengabarkan bahwa proposal diterima dan selanjutnya kandidat yang lolos ke tahapan berikutnya akan dikabari lebih lanjut. Sehingga konsultan tidak menunggu-nunggu berita dari anda dan bisa memutuskan untuk moves on ke pekerjaan lainnya.
Semoga tips diatas berguna ya!
Wednesday, February 10, 2016
Thursday, December 6, 2012
Do we need ethic to work with the local people?
Our way or their way?
Most of development works are done with a set of imported approach including resources and value. We bring something new, something foreign to a specific place, specific communities and hope that they will adopt what ever changes we offer them and thus transform themselves.
And quite often "our way of doing thing" versus "their way of doing thing" are at odd. In my observation, many time we disregard their custom, their system, their way of making decisions, to make sure that we are doing our activities to fit our own standard.
Most of local communities or indigenous communities has system in place that they follow for hundred of years, perhaps since the time of their ancestors.
For example, in rural Indonesia, the tribe community makes decision through communal gathering, most of the time they start with prayer, serve food, go thorough discussion of the topics, and sometimes followed by performing traditional ceremonies.
This process can be rigorous at times and we often left with little patient to go through the process and adopt at their ways of doing the things. Sometimes it's due to efficiency as traditional ceremony can not be part of our budget. Instead we hand-pick those who we consider as the key persons and harvest on their point of views on certain issues and based our project on those who we talked to.
This practice sometimes can hinder our efforts to help the most needy. Those who are marginalized or don't have voice nor influence but to surrender to the decision of others that imposed upon them. And how is this approach in tune with our most clear objective? to target the inequity?
On the other hand, to completely follow the traditions of the local community might not be effective as well, especially when it comes to certain issues like gender and women participation in making decisions, due to the patriarchal system that they adopt.
Tangible change vs intangible change?
It's easy to judge our success by seeing things that are tangible, for example latrines are in place, the water system runs to the village, materials are distributed. But how about things that are beyond our eyes? What has been change in their systems due to our presence? are there changes at all? if so, is it positive? how does this affect the community? and most important things are they going to maintain the positive impact of our intervention and solid as a community?
Framework of ethics to work with the local
It seems, we need a certain framework or guidance on how we conduct our business, especially to work with the local community. And consider if it is important to value their systems. Does it make sense to adopt to their way of doing things and why? and to consider how our intended good will do good than harm?
This reminds me of my journalist friend, Susi. In between our morning tea at work, she asked me a mind provoking questions, why do we need to change the way how the local or indigenous people do their business? they have survived so far, how do we know that we are better than them?
I guess she has a point there, however when it comes to certain aspects like saving lives, helping prevented maternal mortality, helping them to live peacefully in their forests, we probably can offer some interventions.
Most of development works are done with a set of imported approach including resources and value. We bring something new, something foreign to a specific place, specific communities and hope that they will adopt what ever changes we offer them and thus transform themselves.
And quite often "our way of doing thing" versus "their way of doing thing" are at odd. In my observation, many time we disregard their custom, their system, their way of making decisions, to make sure that we are doing our activities to fit our own standard.
Most of local communities or indigenous communities has system in place that they follow for hundred of years, perhaps since the time of their ancestors.
For example, in rural Indonesia, the tribe community makes decision through communal gathering, most of the time they start with prayer, serve food, go thorough discussion of the topics, and sometimes followed by performing traditional ceremonies.
This process can be rigorous at times and we often left with little patient to go through the process and adopt at their ways of doing the things. Sometimes it's due to efficiency as traditional ceremony can not be part of our budget. Instead we hand-pick those who we consider as the key persons and harvest on their point of views on certain issues and based our project on those who we talked to.
This practice sometimes can hinder our efforts to help the most needy. Those who are marginalized or don't have voice nor influence but to surrender to the decision of others that imposed upon them. And how is this approach in tune with our most clear objective? to target the inequity?
On the other hand, to completely follow the traditions of the local community might not be effective as well, especially when it comes to certain issues like gender and women participation in making decisions, due to the patriarchal system that they adopt.
Tangible change vs intangible change?
It's easy to judge our success by seeing things that are tangible, for example latrines are in place, the water system runs to the village, materials are distributed. But how about things that are beyond our eyes? What has been change in their systems due to our presence? are there changes at all? if so, is it positive? how does this affect the community? and most important things are they going to maintain the positive impact of our intervention and solid as a community?
Framework of ethics to work with the local
It seems, we need a certain framework or guidance on how we conduct our business, especially to work with the local community. And consider if it is important to value their systems. Does it make sense to adopt to their way of doing things and why? and to consider how our intended good will do good than harm?
This reminds me of my journalist friend, Susi. In between our morning tea at work, she asked me a mind provoking questions, why do we need to change the way how the local or indigenous people do their business? they have survived so far, how do we know that we are better than them?
I guess she has a point there, however when it comes to certain aspects like saving lives, helping prevented maternal mortality, helping them to live peacefully in their forests, we probably can offer some interventions.
Sunday, June 24, 2012
Likert Scale
Recently I stumbled upon this website regarding the use of Likert Scale, here is another useful website to learn more about this tools.
My reading lead me to this statisticcafeblog which explains furthermore the analysis of likert scale.
I remember my professor said that in order to build an instrument, we need to ensure the validity and reliability of the tools. If I apply that to the likert scale, I would employ some steps:
1. Building the key points that I need to develop by putting up the definition preferably based in available theories or consensus among the experts.
2. Test the content to the experts to provide the validity of the tools and build up temporary classification.
3. Run a factor analysis of the items by sending it out to a number of people
4. Clarify the results and develop the category based on the factors (and subfactors)
5. Run a Cronbach's Alpha Reliability to test the reliability of each category
6. Review and eliminate the items that did not belong to the category.
And the next is to run the actual likert scale to intended responden and analyze the result. Note, if the items are ordinal data avoid to use the mean as this will be useless, use mode instead.
Well, that's the summary of learning today :)
My reading lead me to this statisticcafeblog which explains furthermore the analysis of likert scale.
I remember my professor said that in order to build an instrument, we need to ensure the validity and reliability of the tools. If I apply that to the likert scale, I would employ some steps:
1. Building the key points that I need to develop by putting up the definition preferably based in available theories or consensus among the experts.
2. Test the content to the experts to provide the validity of the tools and build up temporary classification.
3. Run a factor analysis of the items by sending it out to a number of people
4. Clarify the results and develop the category based on the factors (and subfactors)
5. Run a Cronbach's Alpha Reliability to test the reliability of each category
6. Review and eliminate the items that did not belong to the category.
And the next is to run the actual likert scale to intended responden and analyze the result. Note, if the items are ordinal data avoid to use the mean as this will be useless, use mode instead.
Well, that's the summary of learning today :)
Tuesday, June 19, 2012
The complexity of behavior change
Today, I went downtown and met an old colleague. We both used to work for NGO in Indonesia. I mainly worked in the health sectors. Juggling over some programs like HIV/AIDS, Watsan, Health In Emergency, Psychological Support and a bit of blood donors. Together we used to be member of health working group.
Now after 5 years leaving NGO, and met again somewhere else, it was really interesting to look back at what happened back then. I guess to look back after having a distance to something that we used to do, to environment we used to live in or to situation we used to engage somehow provide us with the opportunity to assess it in a more balance manner.
What I want to discuss here is about the approach that we took in doing our work. When we came to the community who needs our assistance, we ensure from the beginning that they are involved in identifying their problems, there are various tools that can be used here VCA/PRA/PPP, etc.
Through this process a set of problems are identified. When it come the communities, the problems rarely simple but it rather complex and mostly one things led to the other. So I would call this a systematic problems. For example, from the field survey we get the information that diarrhea is the highest incidence in a village, this information most of the time confirmed by the data from the local Health Care Unit (if the statistic is available). Then another problem is that most of the villagers do not have access to clean water, we probably assumed that their hygiene practice is poor. So this is great! Now we really have a good reasons to assist them with installing the water system.
And moreover, most of our donors earmarked their funding to Water and Sanitation. So let say that our findings matched our funding, or it can be the other way around, we found a legitimate reasons to write a proposal based on this problems.
So we are happily work in the field, do some surveys, check the water sources, put on some budget and calculation and massively conducted hygiene training to the local people. Cover as much people as possible in a quite short or a lengthy of time.
This involves at least three day training, talking a great deal about personal hygiene, F-Diagram, washing hands, etc. Now the question is what is the impact? On the long term, we would like to see sustainability. But it takes sometime to reach that stage.
What strikes me is after this three days training, lot of lectures, interactive learning, role play and etc then when it came to dining time, still most of the training participants still stick to their old habits, one is taking food with hand without washing it. While washing hands practice is one of our biggest goal out of the training, many times we do not see an immediate impact of this training. Which led to some questions, do they get the message? Do they get overwhelmed by the information given to them through the training? or what is actually makes people change their behavior?
Behavior change for sure does not happen over a night and perhaps we should investigate more about what is the most appropriate approach to encourage behavior change.
Now after 5 years leaving NGO, and met again somewhere else, it was really interesting to look back at what happened back then. I guess to look back after having a distance to something that we used to do, to environment we used to live in or to situation we used to engage somehow provide us with the opportunity to assess it in a more balance manner.
What I want to discuss here is about the approach that we took in doing our work. When we came to the community who needs our assistance, we ensure from the beginning that they are involved in identifying their problems, there are various tools that can be used here VCA/PRA/PPP, etc.
Through this process a set of problems are identified. When it come the communities, the problems rarely simple but it rather complex and mostly one things led to the other. So I would call this a systematic problems. For example, from the field survey we get the information that diarrhea is the highest incidence in a village, this information most of the time confirmed by the data from the local Health Care Unit (if the statistic is available). Then another problem is that most of the villagers do not have access to clean water, we probably assumed that their hygiene practice is poor. So this is great! Now we really have a good reasons to assist them with installing the water system.
And moreover, most of our donors earmarked their funding to Water and Sanitation. So let say that our findings matched our funding, or it can be the other way around, we found a legitimate reasons to write a proposal based on this problems.
So we are happily work in the field, do some surveys, check the water sources, put on some budget and calculation and massively conducted hygiene training to the local people. Cover as much people as possible in a quite short or a lengthy of time.
This involves at least three day training, talking a great deal about personal hygiene, F-Diagram, washing hands, etc. Now the question is what is the impact? On the long term, we would like to see sustainability. But it takes sometime to reach that stage.
What strikes me is after this three days training, lot of lectures, interactive learning, role play and etc then when it came to dining time, still most of the training participants still stick to their old habits, one is taking food with hand without washing it. While washing hands practice is one of our biggest goal out of the training, many times we do not see an immediate impact of this training. Which led to some questions, do they get the message? Do they get overwhelmed by the information given to them through the training? or what is actually makes people change their behavior?
Behavior change for sure does not happen over a night and perhaps we should investigate more about what is the most appropriate approach to encourage behavior change.
Monday, September 27, 2010
Some Facts and Misconceptions about People with Disabilities
Recently, I participated in a training with my local Red Cross about emergency preparedness for people with disability. I found the facts in US quite surprising, some are listed below:
- People with disabilities compromise more than 19% of the population
- Among those who are 65 years old, 52% have a disabilities
- 6% of the population use a wheelchair or ambulatory aid
- Nearly 6.5% of the population have a sensory disabilities
- 7% of the total population have problems with cognitive, mental or emotional functional.
- Society must take care of people with disabilities
- People with disabilities are brave
- People with disabilities undergo a sad life and we should feel sorry for them
- People with disabilities are sick, hurting, fragile and unintelligent
- People with disabilities are sensitive and do not like to talk about their disabilities.
- Discrimination
- Prejudice
- Tension
- Fear
- Ignorance
- Intolerance
- Stereotyping
- Insensitivity
- Discomfort
- Shame
- Avoid using the term "handicapped" , "cripple", "victim", "invalid", "special" instead use the person first langguage "person with disability"
- Instead of using the term normal people use the term people without disability
- Instead of using the term "deaf and dumb, mute use the term Hearing impaired, person who is deaf
- Name the disability rather than called it diseases.
- Don't panic
- Do not assume or segregate
- Treat as intelligent person
- Speak directly to the individual with disability
- Identify your self
- Offer or ask if the person need assistance
- Respect personal property and space
- Be patient and polite.
Thursday, September 23, 2010
Seeing is believing through a new picture of health.
I have been quite fortunate that on the last minute a friend of mine forwarded an invitation to a film screening " A New Picture of Health" held at the National Geographic down town DC.
The program went well, it was host by UN Foundation with several key speakers; including Ms. Joy Phumaphi, the executive secretary of African Leaders for Malaria Alliance, Mr Timothy Wirth, the president of UNF, Ms Natasha Bilimoria, the president of the Global Fight.
Perhaps what is so appealing about the event to me is that this kind of documentary will be a new way of reporting in near future. Reporting is sometimes not only about graphs and charts and number but also about seeing what is actually happening in the field, not necessarily from the eyes of the program implementer but also from the eyes of the beneficiaries. Especially to answer the question (1) Does the program relevant with their needs, (2) Does the program help them to change their situation thus contributed to achieve whatever they decided to achieve and (3) most important is does it enable them to help themselves at the end of the chain?
I came up to the screening with those 3 questions in my mind. More precisely what has been the impact of the work of the Global Funds to the live of the people that are affected by malaria, HIV/AIDS and Tuberculosis ( three top diseases that also inclusive to the MDG's)
The movie captured the stories from three (3) countries; Rwanda, Eithopia and Indonesia. In Adama Eithopia, where a health clinic is treated HIV/AIDS patients, the movie captured the activities of some men, they are are putting up a construction. The treatment that they received enabled them to be healthy enough to go about their daily business; make furniture, tends farms and raising stock. Certain amount of income that they earned is shared to the clinic.
In Rwanda more than a thousands doctors treating 79.000 patients at 400 clinics across the nation. The integration of treatment of this three diseases seemed to be effective in a way that the patients tends to acquired more than one diseases at the same time.
While in Indonesia, Ketut Budiarsini who was diagnosed with HIV while she was pregnant gave birth to a healthy child and she is healthy enough to be able to raise her and continue received treatment from the clinic.
These view immediately answers my previous questions. The global funds program in a way has restored the dignity of human lives. What is so special about the global funds is unlike most programs of which the funding resources is 100% from donors, this program make the participating countries invest 30% of the total funds. This allows greater share of responsibilities among the stakeholders. Overall the Global funds has achieved three important outcomes (1) Rising economies, (2) Healthier Communities and (3) Building partnerships.
At the end I agree with Ms Phumaphi remarks about development program. It is about people, it is about improving the quality of life and it's improved when they owned the development process.
The program went well, it was host by UN Foundation with several key speakers; including Ms. Joy Phumaphi, the executive secretary of African Leaders for Malaria Alliance, Mr Timothy Wirth, the president of UNF, Ms Natasha Bilimoria, the president of the Global Fight.
Perhaps what is so appealing about the event to me is that this kind of documentary will be a new way of reporting in near future. Reporting is sometimes not only about graphs and charts and number but also about seeing what is actually happening in the field, not necessarily from the eyes of the program implementer but also from the eyes of the beneficiaries. Especially to answer the question (1) Does the program relevant with their needs, (2) Does the program help them to change their situation thus contributed to achieve whatever they decided to achieve and (3) most important is does it enable them to help themselves at the end of the chain?
I came up to the screening with those 3 questions in my mind. More precisely what has been the impact of the work of the Global Funds to the live of the people that are affected by malaria, HIV/AIDS and Tuberculosis ( three top diseases that also inclusive to the MDG's)
The movie captured the stories from three (3) countries; Rwanda, Eithopia and Indonesia. In Adama Eithopia, where a health clinic is treated HIV/AIDS patients, the movie captured the activities of some men, they are are putting up a construction. The treatment that they received enabled them to be healthy enough to go about their daily business; make furniture, tends farms and raising stock. Certain amount of income that they earned is shared to the clinic.
In Rwanda more than a thousands doctors treating 79.000 patients at 400 clinics across the nation. The integration of treatment of this three diseases seemed to be effective in a way that the patients tends to acquired more than one diseases at the same time.
While in Indonesia, Ketut Budiarsini who was diagnosed with HIV while she was pregnant gave birth to a healthy child and she is healthy enough to be able to raise her and continue received treatment from the clinic.
These view immediately answers my previous questions. The global funds program in a way has restored the dignity of human lives. What is so special about the global funds is unlike most programs of which the funding resources is 100% from donors, this program make the participating countries invest 30% of the total funds. This allows greater share of responsibilities among the stakeholders. Overall the Global funds has achieved three important outcomes (1) Rising economies, (2) Healthier Communities and (3) Building partnerships.
At the end I agree with Ms Phumaphi remarks about development program. It is about people, it is about improving the quality of life and it's improved when they owned the development process.
Subscribe to:
Posts (Atom)