amazon web services - AWS EC2 (batch) throttling connection? -


so running script on aws batch fetches 100,000+ domains.

  • the script runs in docker container.
  • the domains randomized , in redis queue.
  • the script pulls 20,000 domains queue, processes them , writes results redis again.

when run script on ec2 instance launched get.

ami: custom ebs optimized: true  time elapsed : 2007.7330884933472 domains : 53517 processed domains: 240000

when run on ec2 instance spun batch get:

ami: amazon linux ami 2017.03.e x86_64 ecs hvm ebs optimized: false  time elapsed : 2313.34757232666 new domains : 51243 processed domains: 400000

are instance's aws batch spins throttling connection? because using docker can't think of why results different other bandwidth issues. docker image stored in ecs , pulled down , script run.

i have run test on millions of randomized domains , results same statistically speaking not sampling of domains either. domain rate 2x on instance spin vs batch.

update 1: differences, ebs true on 1 , false on other. can't seem change aws batch.

update 2: tested ebs false machine lower specs , still can't account lower network performance. maybe ami?

update 3: tested ami , problem ami-c6f81abe used on batch. not sure why yet.

update 4: turns out ulimit parameter on jobdefinition causing problem.


Comments

Post a Comment

Popular posts from this blog

neo4j - finding mutual friends in a cypher statement starting with three or more persons -

Image
i trying build cypher statement neo4j know 2-n starting nodes name , need find node (if any) can reached of starting nodes. at first tought similar "mutual friend" situation handled (start1)-[*..2]->(main)<-[*..2]-(start2) in case have more 2 starting points around 6 know name. so puzzled how can include third, fourth , on node cypher able find commmon root amongst them. in above example neo4j website need path starting 'dilshad', 'becky' , 'cesar' check if have common friend (anders) excluding 'filipa' , 'emil' not friends of three. so far create statement programmatically looks like match (start1 {name:'person1'}), (start2 {name:'person2'}), (start3 {name: 'person3'}), (main) (start1)-[*..2]->(main) , (start2)-[*..2]->(main) , (start3)-[*..2]->(main) return distinct main but wondering if there more elegant / efficient way in cypher possibly use list of names parameter ...
Read more

php - How to remove letter in front of the word laravel -

Image
based on picture above result of var_dump() , dd() of same variable ,when var_dump() , apostrophe (') symbol in black diamond of question mark , when dd() same variable , apostrophe can seen letter 'b' appear in front of words what want can me result : loreal's sdn bhd and @ same time want remove special character ( excluding these 4 symbols (-) ,(_) ,('),(,) ) simply escape special character resolve issue. mysqli_real_escape_string resolve issue. $name = mysqli_real_escape_string($request()->user); or use addslashes function() $name = addslashes($request()->user);
Read more

minify - Minimizing css files -

i want know how css files can minimized. currently, using external sites css minifier compress css files. however, after removing whitespaces, comments, etc, file sizes reduced 10-15%. how frameworks bootstrap manage make minimized versions kilobytes. as example, css is: body { background-color: #ff0; } the reduction goes 28 bytes 22 bytes other css files have lot more characters still smaller this. actually minifying code means it's remove thing code white space characters new line characters comments block delimiters it reduces amount of code has transferred on web , speed site load fastly.but unreadable format.
Read more