How to (temporarily) stop Elasticsearch from expunging deleted documents? -


in my usecase i'm trying synchronize 2 elasticsearch indices. due versioning acutally quite simple. however, want keep writing @ time while i'm doing this.

okay, steps want perform in chronological order:

  1. clients write (index, delete, update) cluster c1
  2. create new index c2 (clients keep writing c1)
  3. copy data cluster c1 c2 (clients keep writing c1)
  4. switch clients c2
  5. synchronize changes c1 c2 (clients keep writing c2)
  6. shutdown c1

step #5 step i'm looking at. have make sure changes written c2 aren't overwritten data c1. using versioning it's rather simple writes index operations fail (versionconflictengineexception). assuming following situation:

  1. a document updated on c1 right after #3 (v2 on c1, v1 on c2)
  2. the same document deleted right after #4 (v2 on c1, deleted on c2)
  3. synchronizing try reindex v2 on c2

i know elasticsearch keeps deleted documents around while:

# index document 1:4 $ curl -xput 'http://localhost:9200/test/test/1?version=4&version_type=external' -d '{"message": "test"}' {"ok":true,"_index":"test","_type":"test","_id":"1","_version":4}  # delete document 1:6 $ curl -xdelete 'http://localhost:9200/test/test/1?version=6&version_type=external' {"ok":true,"found":true,"_index":"test","_type":"test","_id":"1","_version":6}  # index document 1:4 (error!) $ curl -xput 'http://localhost:9200/test/test/1?version=4&version_type=external' -d '{"message": "test"}' {"error":"versionconflictengineexception[[test][2] [test][1]: version conflict, current [6], provided [4]]","status":409}  # wait time  # index document 1:4 (success!) $ curl -xput 'http://localhost:9200/test/test/1?version=4&version_type=external' -d '{"message": "test"}' {"ok":true,"_index":"test","_type":"test","_id":"1","_version":4}

the problem "wait time" part. have rely on deleted documents unknown amount of time. therefore need control time disallowing expunging of deleted documents while i'm running #5. how this?

crossposted elasticsearch group (and related question)

the "wait time" part defined index.gc_deletes setting on per index basis , set "60s" default. can dynamically changed using update index settings command.


Comments

Post a Comment

Popular posts from this blog

curl - PHP fsockopen help required -

i trying work on cloudflare api , have found in documentation. unfortunately have no idea how can use in fsockopen or through curl. here example of post request. need know how can make request below data post using curl or fsockopen curl https://www.cloudflare.com/api_json.html -d 'a=cache_lvl' \ -d 'tkn=8afbe6dea02407989af4dd4c97bb6e25' \ -d 'email=sample@example.com' \ -d 'z=example.com' \ -d 'v=agg' here link cloudflare https://www.cloudflare.com/docs/client-api.html#s4.2 try one: $ch = curl_init("https://www.cloudflare.com/api_json.html"); curl_setopt($ch, curlopt_returntransfer,1); curl_setopt($ch, curlopt_followlocation, 1); curl_setopt($ch, curlopt_ssl_verifypeer,false); // handling of -d curl parameter here. $param = array( 'a' => 'cache_lvl', 'tkn' => '8afbe6dea02407989af4dd4c97bb6e25', 'email' => 'sample@example.com', 'z' =...
Read more

HTTP/1.0 407 Proxy Authentication Required PHP -

i working in restricted network/limited access internet. since have use proxy make web service call. here php code calling web service , response . $client = new soapclient("http: //www.xyz.com/abc/wsdls/login.wsdl", array("trace" => 1, "exceptions" => 1, 'proxy_host' => $proxy_host, 'proxy_port' => $proxy_port, 'proxy_login' => $proxy_login, 'proxy_password' => $proxy_password)); response is fatal error: uncaught soapfault exception: [wsdl] soap-error: parsing wsdl: couldn't load ' http://www.xyz.com/abc/wsdls/login.wsdl ' : failed load external entity if copy url error message , paste in browser(with same proxy settings) works. 2nd scenario is, if pass values instead of variables giving me diffarant response $client = new soapclient("http: //www.xyz.com/abc/wsdls/login.wsdl", array("trace" => 1, "exceptions" => 1, 'proxy_host' ...
Read more

c# - Resource not found error -

i have problem accessing method on homecontroller. show code of method : [httpget] public actionresult decriptidentifiantetredirige(string login_crypter, string mdp_crypter) { string loginacrypter = _globalmanager.protegemotdepasse(login_crypter); string mdpacrypter = _globalmanager.protegemotdepasse(mdp_crypter); user userapp = new models.user(login_crypter, mdp_crypter); if (userapp.authentificationvalidee(userapp.userlogin, userapp.password)) { session["name"] = userapp.userlogin; return redirecttoaction("accueil", "home"); } else { return redirecttoaction("validerauthentification", "home"); } } then in routeconfig.cs wrote route : routes.maproute( name: "authentificationapresdecryptage", url: "{controller}/{action}/{login_crypter}/{mdp_crypter}", defaults: new...
Read more