Open Search Engine development and maintenance

in #hive4 years ago (edited)

This proposal is a request for funding to pay for the development and maintenance of search engine that provides search functionality to hive.blog, peak.com, ecency.com and other potential Hive apps.

Project Title

Open Search Engine development and maintenance

Blockchain-based-Open-Search-Engine

Proposal Type

Hive service - Opensource

Costs

Daily: 45 HBD
Monthly: 1350 HBD


Labor (Development+Management): ~25 hours per month, 750 HBD
Recurring costs (Servers): ~$600, ~600 HBD

Progress reporting: Bi-Weekly

Project Summary

Project description

Search engine is based on Elasticsearch and it is best in one thing, indexing and searching content. Current version of Search Engine is working good but we want to improve it so content ranking and indexing works better within community searches, optimize fields, add check for efficiency and of course we want to provide Open API access to all who wants to build on Hive.

Home: https://hivesearcher.com
Github: https://github.com/ecency/hive2elastic

Benefits

Right now, Search engine is used by all major frontends, Hive.blog, Peakd.com and Ecency.com and to cover part of the server costs, we have monthly payment for API key and plans with limited access. We want to remove those limits and open it up for every app to integrate. https://search.esteem.app/api-register
There are potentially unlimited opportunities for developers to increase exposure or create new types of apps, content discovery could be improved by different creative approaches.

What happens if the proposal becomes unfunded?

In case proposal becomes unfunded, we will continue to have monthly fees or increase fees to cover server cost with not much development. It will likely end up being non-sustainable and every app creating their own private versions costing even more and unstable user experience. And difficult to potential new developers/apps to integrate and use.

Explanation of proposal costs

In terms of development, as mentioned above, we will improve indexing and ranking, introduce new filters. New indexing will increase server costs as Elastic search will need new cluster and more storage to handle indexes. We are creating this proposal for just 6 months. After 6 months, we will create new proposal with smaller amount just to cover server costs. We think, by then not much new development will be needed.

Support this proposal

Vote for this Proposal using Hivesigner
HiveDao: https://hivedao.com/proposal/114
PeakD: https://peakd.com/proposals/114
HiveBlog Wallet: https://wallet.hive.blog/proposals
Github: https://github.com/ecency/hive2elastic

Progress update

Sort:  

I would like better search functionality on Hive. Looking at the repository https://github.com/esteemapp/hive2elastic, I don't see any mention of a license.

An open source license is a prerequisite for me to support this work. Personally, I think the BSD-2-Clause Plus Patent License would be the best choice for your Python codebase, but any common permissive open source license would probably be sufficient.

Without a license, the community is not able to reuse and build off your code.

Thank you, agree somehow we forgot to add license to this repo. As all our work is usually MIT license, I just added MIT license to this as well.

Great! I see the license added in b696736.

Will happily support this proposal now.

One common search engine will be really great and especially a public API consumable for other DApps will be awesome. 👍

Search is a must have!

Synchronistic! I also proposed some improvements to our search engine just yesterday. I see that our current system is working in some respects, but not in others. For example, I see that the search results are stacking up comments equally to posts, but that doesn't make sense because ~90% of the comments are not informative (the real information is in the articles usually). To me, comments are not equal to articles, even thought they are equally labeled "posts."

So, in addition to your proposal for improvements to the search functionality, please add the ability to exclude comments from the rankings, or at least priorities entire articles over simple comments. I support the idea of adding advanced filters, so we can organize search results based on timing, credibility (upvotes), and authority (reputation).

Great. Always wanted a better search engine. Let's support this proposal. It can be very daunting when you are trying to find your own article and it pops up with articles which doesn't even bear the title you are looking for

That's what we are going to improve.

I'll support it. Ty for your efforts to make Hive better.

Thank you for your support! 🙇

Right now it's a bit confusing: when searching for keywords the oldest posts come first. However, since STEEM/HIVE evolves rapidly, posts are outdated. It would be better if the most recent posts including the searched keywords appear first.

For example, when searching for posts on how powering down works, writing "powerdown" will give results of posts in 2016 when a powerdown took one year.

Upvoted the proposal.

There are few filters in place and UI should give option or they can even introduce their own sorting. We will be adding more filters as a part of further development

That's a good step

This is something that has been quite needed for a while.

I appreciate the format and financial cost breakdown provide in this post.

I have supported this. The cost seems rather reasonable.

Great minds are really working here on hive. Better days ahead @good-karma

This should be top priority! The broken search on Hive made me loose so many new members, and I had been messaging random witnesses about it to make sure they were aware.
So thank you for doing this!

Great initiative

That would be useful. Thanks @good-karma :)

I'm supporting and voting for this project.

Voted! Thanks for help to desenvolve the hive experience!

That's great, if we can have a great Search Engine, it would be a great addition to the number of dApps we use.

very interesting

I'm very interested in this project, search engine is a necessity

What we currently have for searching articles within the blockchain is nothing short of horrendous. I 100% support this project. Thank You & Best of Luck!

Thanks for great info

Hola @good-karma. Gran iniciativa, muy interesante. Saludos desde Colombia.

Hello @good-karma. Great initiative, very interesting. Greetings from Colombia.

Perfect! Supported! I find the HBD ask appropriate for such a needed service!

In the past I had some mixed result using your search engine but I trust you and your ability to improve the system. If you put some work into the indexing, the search engine will start returning more relevant search outcomes for sure.

Amazing work! One of the easiest proposal approvals ever for me. Thanks for all the hard work 💪

You have my support: an efficient Search Engine is fundamental, isn't it?
!tip !BEER 🍻 and a huge hug! 🤗

Good step taken by the way

Go0d step by the way