There were no official prices on the twitter pages for the search endpoints and the powertrack apis. I tried to inquire and that is when I found out about the pricing changes (I am @blackforestboi on Twitter).
There were some prices mentioned about the Account Activity API which were also quite pricey already. I think something like $2500 to follow updates of 500 accounts or so.
The key difference is that most APIs consumers and projects used were the v1.1 and v2 apis which were free to use with rate limits that were actually quite generous.
(tweets, timelines, messages, bookmarks, lists, followers etc)
Now those APIs are all behind that enterprise paywall with that entry price of $42.000
We built Memex for that combination of use cases in mind.
So right now you can already full-text search your bookmarks, and filter by time, domain and tags. Already on the mid-term roadmap we plan to enable full-html/text snapshots of visited pages, both locally and on-demand. For the latter we are potentially working with the Internet Archive.
Yeah indeed, having stuff in the cloud is not ideal when it comes to privacy and centralisation.
One of our core values is privacy and data ownership. So we do our best to make our business not dependent on (analysing and selling) your data, and instead provide you with service value you're willing to pay for.
We are built with interoperability in mind, that will allow you to switch providers of Memex and Memex Cloud without frictions, in case there are breaches of trust, or simply better service.
We follow these values by currently building for offline first usage, where your data is locally indexed and searchable primarily.
With our search technology you'll be able to get up to 5 years of your research done in the browser.
For the cloud part, it is unfortunately not yet possible to do performant search on encrypted data, otherwise it would not be such a big issue to have your index in the cloud. Equally unfortunate is that it comes with a lot of drawbacks to replicate all your data on all nodes, as opposed to have a central point to query. Especially when we are looking at phone usages.
There it is really not practical, so there is a need to have some sort of cloud - UX is still very important. Most people can't be bothered with the drawbacks of decentralised and distributed systems (yet). We hope to get that switch in multiple smaller steps that guide (non-technical) users through a smooth transition to a Memex system that is as distributed as possible. (Check out Dat https://datproject.org/, a technology we likely use to make that first step possible)
And as you already noted, this stuff will be self-hostable. We see ourselves as a service provider first and want to serve people who can't/don't want to run their own server.
A bit like the Wordpress model.
You can read more about our approaches to running this business in our vision post: worldbrain.io/vision
I was wondering about mobile usage too, and agree that having a server somewhere available for queries is the best solution.
Since this deals with such private data, having a possibility for self-hosting is the correct solution.
Good news :)
This tool is open-source, and runs fully locally (except, as pointed out the feature that enables you to share quotes, because for that stuff you unfortunately need a server, still)
We custom built a search technology on IndexedDB and Dexie.js, which is capable of indexing around 5 years of your personal web-research locally in the browser.
Yeah indeed, just full-text search can let you end up with a lot of garbage.
This is why it is so important, that you can search for various other "vague memories" to narrow down your search.
What you often remember about an article is stuff like: Did I bookmark it, when did I visit it, did I like/share/cite it on social media
You can already filter by time, tags, domains, bookmarks, and soon also if you liked/shared or even seen it in your newsfeed, or on a friends wall, on Twitter and Facebook.
We gradually expand it so you can search with as much of your associative memories as possible.
Oli here, from the team developing Memex.
Memex is open-source, so the browser extension will always stay free to use.
What we will charge for are some of the services that require us to host stuff. Like backups, multi-device syncing, API calls etc. We will run it as a completely modular pricing model, where you can upgrade on only those features you need. We don't like those usual 3 tier model, where you have to upgrade to the 'monster mega plan' in order to just get one feature :)
But you can also completely self-host that, as we will make the server software open-source as well.
Hope Memex can be useful to you.
We are running a crowdfund to support its development, where we offer some good discount on the future features in return: worldbrain.io/pricing
Yeah as mentioned the tool allows you to do bookmark full-text search.
It's open-source, so it will stay no matter what.
We build for resilience and for us as the WorldBrain.io company not needing to stick around in order for the service to survive. We see ourselves as the stewards of this tool, not the sole benefactor or proprietor.
Firefox opens up their Awesomebar API on April 17 with the new FF53 release.
We are actually actively building the full-text history and bookmarks search as a chrome (and soon also FF extension)
(http://worldbrain.io)
It's also all open-source.
thanks for your comments. Yeah, we are aware that this could become an issue and will work on making it secure in the future.
Thanks for your offer to share your experience. You can write me at oli@worldbrain.io or we have the discussion here for now.
Whatever you are more comfortable with!
I'm curious about the organizational aspect of your project. that's a nut I've yet to crack. what was your strategy in building a community around your project?
First, we want to make this tool really usable to the individual and gather a couple of thousand users with it.
Without the needs of an individual fulfilled, there won't form a "fact-checking" community after all.
We want to spread it by engaging in those groups on the web where fact-based discussions are happening, as well as the general science communication community.
Also providing/gathering more tools and useful content that help our users to do their work more efficiently.
Basically by building a content brand around collaborative web-research, fact-checking etc.
Then in further development, we make the sharing part possible, from which we think that a community will form quite organically.
The key difference is that most APIs consumers and projects used were the v1.1 and v2 apis which were free to use with rate limits that were actually quite generous. (tweets, timelines, messages, bookmarks, lists, followers etc)
Now those APIs are all behind that enterprise paywall with that entry price of $42.000